So… after writing Onward, I say! I spent pretty much all day looking for answers. And I found them. Turns out how I was interacting with the database was sort of wrong, I need to tweak some things:

ta-da!

I was able to maintain a pretty responsive UI while the internals worked as hard as it can to make data available. Then I ran into a threading issue where quite frankly I had to rewrite the parser two times. Most of the rewrites were because I failed to understand how the stream parser worked rather than the source being faulty. Turns out that the threading issue was caused by a while never ending loop due to the parser never moving to the next element so it always gave me the same one.

All in all. I managed to solve the database issue, refactored the mechanism that gets the data and it’s oh so much better now; I managed to remove the complexity of it and now I just do a few conditional checks to see if everything’s okay.

The new parser seems pretty stable, rather… it’s probably the most stable parser I have written… so hopefully I won’t have to do any serious changes.

I think I can finally take a breather…