-
Notifications
You must be signed in to change notification settings - Fork 897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory-efficient zlib usage across Liberty file consumers #4834
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've only reviewed gzip.cc
/gzip.h
, it looks fine except for the nits
int bytes_read = Zlib::gzread(gzf, buffer, buffer_size); | ||
if (bytes_read <= 0) { | ||
if (Zlib::gzeof(gzf)) | ||
return traits_type::eof(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading https://en.cppreference.com/w/cpp/io/basic_streambuf/underflow we might need to do something about "On failure, the function ensures that either gptr() == nullptr or gptr() == egptr." because the invariant of when this function is called is weaker, it is "The public functions of std::streambuf call this function only if gptr() == nullptr or gptr() >= egptr()."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch, now I setg(eback(), egptr(), egptr());
before returning eof
93c6d2d
to
b85dda9
Compare
In ASIC flows, Liberty files provided in the PDK can be very large and are often compressed with gzip. Prior to this PR, the only pass that accepted
.lib.gz
wasread_liberty
, just likeread_verilog
accepts.v.gz
etc.-liberty filename
arguments to passes likeclockgate
anddfflibmap
didn't use the decompression code.The
decompress_gzip
function also has been reading the entirety of the uncompressed file into RAM (into a stringstream). This is now fixed by implementing angzip_istream
much like the pre-existinggzip_ostream
. It has one byte of guaranteedunget()
which is sufficient for our Liberty parsing. Using gzstream was ruled out because it's LGPL.The PR carries also some shuffling of things around which should have limited impact on plugin devs since the new
kernel/io.h
header is included inkernel/yosys_common.h
.This will resolve #4830 but only on the Yosys side. The Yosys
abc
pass only hands off the paths specified withabc -liberty path
toabc
butabc
doesn't support.lib.gz
which is surprising since it does have its own vendored zlib and wrapper around it used for aiger