Ugarit performance (by )

Ugarit was once renowned for its poor import performance, and rightly so. However, it's a lot faster these days - not, sadly, due to amazing optimisation work on my part, but because Thomas Hintz made write-u8vector! faster. It's not released yet, but will be in Chicken 4.10.

There's still work to be done, though. In my experiments with archival mode, I imported 9GiB into an archive in:

real    24m14.822s
user    17m51.485s
sys     1m59.920s

Writing an uncompressed tarball of the same 9GiB took:

real    8m49.931s
user    0m1.076s
sys     1m1.315s

That's a factor of 3. Ugarit spends four minutes waiting for I/O while tar spent eight minutes, which is puzzling, but Ugarit spent seventeen minutes of CPU time while tar spent one second; this will be down to the fact that Ugarit still copies each byte of the file several times between reading it in and writing it out, and I know how to fix that!

There will be some unavoidable architectural cost in the fact that Ugarit will always use at least two processes - a frontend and a backend, with the data sent over a pipe between them - but I think there's a lot I can reduce first. Onwards and downwards!

2 Comments

  • By Dakota, Fri 24th Apr 2015 @ 8:15 pm

    Awesome!

  • By @ndy, Mon 11th May 2015 @ 10:28 am

    Maybe the pipe can be zero-copy on operating systems that support splice?

    How much do you thrash the GC whilst doing those copies?

Other Links to this Post

RSS feed for comments on this post.

Leave a comment

WordPress Themes

Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales
Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales