Ugarit performance (by alaric)

Ugarit was once renowned for its poor import performance, and rightly so. However, it's a lot faster these days - not, sadly, due to amazing optimisation work on my part, but because Thomas Hintz made write-u8vector! faster. It's not released yet, but will be in Chicken 4.10.

There's still work to be done, though. In my experiments with archival mode, I imported 9GiB into an archive in:

real    24m14.822s
user    17m51.485s
sys     1m59.920s

Writing an uncompressed tarball of the same 9GiB took:

real    8m49.931s
user    0m1.076s
sys     1m1.315s

That's a factor of 3. Ugarit spends four minutes waiting for I/O while tar spent eight minutes, which is puzzling, but Ugarit spent seventeen minutes of CPU time while tar spent one second; this will be down to the fact that Ugarit still copies each byte of the file several times between reading it in and writing it out, and I know how to fix that!

There will be some unavoidable architectural cost in the fact that Ugarit will always use at least two processes - a frontend and a backend, with the data sent over a pipe between them - but I think there's a lot I can reduce first. Onwards and downwards!

Ugarit archive mode manifest maker (by alaric)

When I last wrote about Ugarit progress, I had developed archive mode to the point where one could import a list of files with metadata from a "manifest file", and then search for files based on the metadata from the manifest and stream out chosen files. I gave an example of using this to play MP3s matching a search pattern:

[alaric@ahusai ugarit]$ for i in `ugarit search test.conf music '(= ($ artist) "UNKLE")' keys`;
do ugarit archive-stream test.conf music $i | mpg123 -;
done

Well, that was all based on hand-written manifest files, which are no fun to produce (our music collection is large). As such, I've been working on a "manifest maker" that takes a list of files and directories and makes a manifest file from them, recursing down through directories to list all the files. And for each, it automatically extracts metadata into the manifest file, which can then be hand-edited if required, and then used to import from.

The idea is that the manifest maker will have support for a number of file types it knows how to extract additional metadata from, and the first one I've implemented is ID3 tag extraction from MP3s. I've implemented the ID3 V2.2 and ID3 V2.3 specs, as those were the two that I found present in the subset of my MP3 collection I'm testing against!

For example, here's the output it produced for one of my MP3s:

(object "./test-data/THE HOLLIES - He Ain't Heavy, He's My Brother.mp3"
  (filename = "THE HOLLIES - He Ain't Heavy, He's My Brother.mp3")
  (mime-type = "audio/mpeg")

   ;; Unknown ID3 tag "COMM"="engiTunNORM\x00 00000402 00000000 00001B59 00000000 00004E65 00000000 000040EC 00000000 00015FD5 00000000"
  (keyword = "Pop")
  (name = "He Ain't Heavy, He's My Brother")
  (creator = "THE HOLLIES")
  (creation-date = "2002")
  #;(featuring = "")
  (collection-name = "Legends CD2")
  #;(collection-volume = "")
  #;(collection-volumes = "")
  (volume-index = 16)
  (volume-size = 18)

  (mtime = 1428948696.0)
  (ctime = 1428948696.0)
  (size = 4063360))

It prints out unknown ID3 tags as comments, in case a human can glean some useful information from them to put into the metadata, and it suggests the names of metadata tags I might be able to provide by hand that it hasn't found (in this case, a tag for other people featured in the music, and two for indicating that this album is part of a set. As it happens, it is, as the "CD2" in the name suggests, but it wasn't indicated in the ID3 so I'll have to hand-edit it; likewise, the date from the MP3 of 2002 is clearly for the production of the album, not that classic track... ID3 metadata is often a bit shabby!). Also included are file mtime, ctime, and size in bytes.

I hope to add Ogg Vorbis metadata next; I'd like to add EXIF support to parse information out of the JPEGs in our vast family photo library, but it looks much harder, and I'm not sure how useful it will actually be!

Exercise and the Failure to Diet (by sarah)

So today I take Jean swimming, today I go swimming for the first time without needing aid to get into the pool for a decade. Today I am fretting that without my glasses on I will not be able to keep track of my 9 yr old or that due to not having been swimming much I will have forgotten how to and I could only just barely swim as it was.

Today I stand here with two swimming costumes, my new one black and multicoloured and my old one - black and white. One is really too big for me and one is really a little too snug and just fit last week and I've had a pigging out birthday weekend so my stomach is not wanting to be compressed.

I am fretting that one will slip off and the other will leave back dugs - or folds of skin, I am fretting that before Alaric has always been there and now I wont be the obvious wife and mother but just me and I do not look young and beautiful. At the same time I worry because I put the swim suit on and I feel naked and exposed and I'll be in public. I have some pit stubble and some leg hair and I don't want to use my energy in dealing with them as that may result in no swimming. I am only an ok size if I hold my stomach in and it hurts today.

I have the scar on my knee from the 10 yr old biking me and scars from caving mucking up my shin and the varicous vein sticking out and ugly on my thigh, cellulite bobbling and I'll admit I don't look that different to me as a teen and I felt all of this then as I panicked over day glow bikini or my turquoise swim suit.

And I just want to hide but Jean... Jean is growing up and wants to swim, I can take her now - for the first time since being a mum I myself by myself can take her. She is already self concous and worried about her body and I don't know if confiding in her or hiding it from her is best.

Bingo bango bong - it's time talk about my diet and expectations. People think the diet is about weight loss but it's not I've also felt fat, it doesn't matter what size I am I look in the mirror and see the same sized person. For a little back ground on that - this has been the case when I was size 8 to being in the last trimester of pregnancy at a whopping 16 stone. One of the swim suits is an 18 and the others a 10 - to look at they don't seem that different.

My diet has been driven by medical stuff and the want to feel better - of course I want to look better but I resigned myself to being not normal aesthetics a long time ago. This is part of the panic before I go out.

My hair is big, it is frizz, it is curl, my skin is blotchy and changes colour and I have scars and now stretch marks. I have an enormous bottom - always have had, it sticks out and is wide. It's just the way I am.

I like weird clothing but I know from having tried that I still stick out even in normal clothing so I might as well wear what I want.

Recently the diet has kind of failed, I have reverted to the 1600 cals a day and I no longer have a nurse to talk to about this. However, I am now a quantified self and have been monitoring things - not weight Alaric deals with that as I don't really want to know. But I do know that my weight has plateud, which is amazing as I keep eating 200 cals more than my maintain intake. The maintain is what you can eat without putting weight on but you wont loose weight on it either.

And the strange thing is that everyone is now commenting on the weight loss but I am no longer loosing it - what is happening however is that I am going down dress sizes. I am physically becoming smaller or more compact as I exercise.

For me it is strange how the focus is on the weight loss, I assume this is because it is the most easily quantifiable thing?

On the other hand my pedometer game is going really well and today I won a pink lemur, the more exercise I do the more in game energy I have to make plants grown and to build buildings with. I still have huge areas to unlock on my maps and it is my main motivator - except now the headaches are under control and the pelvis is behaving and the bleeding has stopped I am enjoying the exercise - but I wasn't before not why that was all going on - I was just doing it when I could for fear of being unfit/fat except there was no way I could do enough.

I mention this as there are people out there who keep trying to use me as a gauge for themselves - you can't. I couldn't even use myself as a gauge six months ago. Each person has to find out what works themselves and sadly my journey has been made a lot more doable by money. Yes you can exercise cheaply - ie running but without proper shoes you risk shin splints, without the pedometers motivations to actually do a run can be low.

Gyms cost money and adults on the kids climbing frames at the park can end up in the police being called. Swimming costs money, dance, climbing, yoga cost money. Youtube vids are free but can take alot of sorting and again motivation.

I like my allotment and it is exercise but... it cost money and it's not even a council one as I've now been on the waiting list for one of them for 3 yrs! They do not have enough allotments >:( Everyone should have access to an allotment >:(

This post initially was supposed to be about the emphasises being on health and not actual weight loss and how size and weight are a loose corralation and not absolute. But it was high jacked by my nerves about swimming and self identity and stuff.

I have everything packed and ready and I know Jean is uber excited - I've gone with the larger swim suit with the idea that a boob popping out maybe is preferable to being uncomftable and not being able to swim. Only time will tell if I can remember how to actually swim and for that matter get out of the pool by myself at the end of the session.

London (by sarah)

My London, my city
Is falling through the cracks
Slipping down between glass
And chrome
Squeezed into non existence
Becoming darker
Messier
With pretention
And no protection
It's soul is dying
As all the things I knew
Crumble under callipers
And blue prints
Raised for respectability
Losing the unique
And the grotesque that was more
Becoming safe
With edges of dire
My City, My London
Soon will be no more

Experiments in Food: Soylent and Joylent (by alaric)

I was interested to hear about Soylent - the meal replacement, not Soylent Green - when it came onto the scene. I lead a busy life, which includes cooking for two kids and a wife (one of the children has an intolerance to cow milk, and my wife has an intolerance to gluten as well as milder issues with cow milk and soya). I often find myself in the situation of being quite hungry myself, at the start of needing to cook a complicated meal from fresh ingredients. Also, I don't like eating breakfast until an hour after getting up, so I tend to eat in the office on weekdays; due to a shortage of filling breakfast options that will last long enough for me to finish them off on two days a week, I often end up skipping breakfast at weekends or just grazing on raw root veg from the fridge, then feeling woozy come lunch time (about when I need to start organising lunch for the family). So the idea of a powder I can store for a long period and then turn into a balanced meal replacement with near-zero effort, and cheaply at that, certainly has some appeal.

Sadly, Soylent decided they can't ship to the UK (and muttered something about refunding my contribution to their crowdfunding effort on that basis, but not until November 2015 - and I funded them in July 2013!), so I gave up on the idea of giving it a go.

But thankfully, they have published their recipe online, which has prompted a Dutch company to set up shop making it and shipping it from the EU! They're called Joylent, and as the name suggests, are taking a rather light-hearted approach to producing basically the same stuff. So I gleefully ordered some, and have started experimenting with it.

I don't plan on living on the stuff, although some have - I just want an easy, filling, meal replacement for when circumstances require it.

So far, I've had two "meals" of it, and the results have already been somewhat interesting.

The first one was a weekend breakfast replacement. I started with the vanilla flavour; it was tasty and 600ml of the stuff went down easily, leaving me feeling satiated. I found I felt full, and with plenty of energy, but I was craving crispy and strong-tasting food; I think my mind didn't quite believe that a soupy liquid with a gentle vanilla flavour could have actually fed me. Although I was craving salty fries and pickles, I didn't actually want to eat anything; I just wanted those flavours, and would probably have been happy to just nibble a tiny amount or something.

Come 1:30pm, four and a half hours later, I felt a sudden pang of hunger, but it passed quickly. I still didn't feel light-heated and ill as I often do when I can't easily eat. In this particular case, I was on a long drive, so we didn't get to stop for lunch until 3pm. I was feeling... "peckish" by then; I fancied the idea of eating, but wasn't suffering from hunger, which was unusual for such a late lunch. I ate a paneer tikka wrap and the leftovers of Sarah's nachos, so plenty of exciting textures and strong tastes, which was exactly what I wanted! I didn't have to eat very much to feel full and satiated, and had a light (and nutritionally meagre) dinner of chips and ketchup that evening (due to lack of alternatives, being a vegetarian in a place that focussed on the eating of sausages), and went to bed not feeling malnourished at all.

The next morning, I had a whole grapefruit for breakfast, but was feeling pretty hungry come lunchtime at 1pm. Sadly, the place we ate was focussed on the eating of roasted animals, so all I had for lunch was a small plate of roasted potatoes and steamed vegetables, which was tasty but not very nutritionally diverse (I'd eaten little protein since lunch the day before). So before setting off on the drive home, somewhere in the middle of the afternoon, I had a second vanilla Joylent. It was pleasing that I'd been able to chuck my nice Joylent mixing bottle and the open pack in my bag for the trip; I bought a bottle of water in a shop to mix it up, but if I'd brought my own bottle of water I'd have been ready to throw together a "meal" wherever and whenever I wanted.

I was once again satiated, although a little less so; I think I put less powder in (judging a third of a pack of powder is tricky, although I think I'll soon be able to work out how big a third of a pack looks in the mixing bottle and get it right in future), as it came out a lot more watery this time, despite not being full quite to the 600ml mark. Once again, I was quickly craving crunchy strong tastes, so at about 8pm, I ate some salt and vinegar crisps at a motorway service station. This quickly led to me feeling I'd eaten too much salt; I felt a bit dehydrated and had a nasty salty taste lingering in my mouth. I suspect the morale of that story is that I've become accustomed to eating too much salt; I need to train my mind to realise that I don't need to have tasted savoury salty flavours to have eaten a meal!

As I lie in bed typing this at 11:15pm, I'm feeling a bit hungry, but not uncomfortably so (I didn't have dinner or anything else to eat).

The Joylent flavours are banana, chocolate, strawberry and vanilla; I think it would probably be a good idea for me to train myself out of craving salty tastes in a meal, but it's quite interesting that I've not found anything else I've eaten lately as filling as the Joylent, nor able to keep me "going" as long. I could see myself living happily on the stuff, but I would really miss food tastes and textures. However, it's made me more aware of how nutritionally limited a lot of foods are. I like the thought of using something like this as "fuel" and then having small quantities of spicy crunchy foods for the taste!

Also, it would be interesting to try and make a curry flavoured Joylent. Either get some without any flavourings added, or start with a mild-tasting one and blend in a nice mix of spices. I may have to perform some experiments in that area!