Chasing the butterflies in GDB

You know when you are chasing bugs it can take a lot of time and energy. While I’m still fatigued I ran into a small design problem, a decision I wrote without really thinking how QNetworkReply would react thus I ran into a lot of crashes. See, crashes you at least have info to work with as the GDB would stop and deliver the frames that you will later check.

When it comes to CPU hog though, everything becomes a chase. Sometimes you either require gut feeling, or a lot of poking through the threads to see if a thread is stuck in a loop. Sadly, due to my limited experience with GDB I just can’t say “hey, alert me if you are busting a nut over there”.

Before I go on, the reason I ran into this specific problem was because I’m taking advantage of QObject parent/child relation. At the same time I’ve been moving away from pointers or anything heap related unless I really require them, which so far I haven’t had the need. This also means less memory management for me. So, if a QObject that is a parent dies, it takes the children along with it, which is incredibly ideal for me.

See, these butterflies I’m talking about aren’t real. I made the mistake of doing a test with data that is 10 years old, with address that I have to poll with QNAM. There’s a lot of redirection/error handling that needs to be in place–which I have taken care of–so I noticed that a thread (apparently a HTTP Thread by NAM) was hogging a core for itself, then after 30 seconds to a minute it would stop/terminate. My first thought was to poke the parsers, since last time I ran into a loop problem as they weren’t able to handle a instruction so it went into a loop. I think, maybe some wouldn’t call it CPU hog as to be honest said CPU hog stopped.

I got fed up; it was a dead end. If there’s a problem with my source code, I can’t detect it. I’m flushing QNetworkReplies as safely as I can, aborting them as safely as I can. So… time to go with a fresh test case. I’m using recent URLs of updated sites and have noticed that there’s no CPU hog anymore. Granted, I still have to keep testing.

I feel like I wasted a day though. A dead end with no clue. It’s a bit frustrating as all these speculations of a memory/cpu hog problem wasn’t real, or maybe it is and I haven’t really scratched the surface of it. On the bright side I cleaned up an class and its behavior is more predictable than before.

On a side note, I feel like I’m doing premature optimization and I should really stop. There has to be a legitimate issue before I jump into these types of searches.

The weird case of proxy model

Rest in peace, David Bowie

***

So basically I spent the day working with Qt’s proxy model. I didn’t get anything done, literally. I ran into a really weird problem, the proxy model I was working on wasn’t propagating methods like data. I added them both in the source code and header, nada, nothing! I’m baffled with this behavior, my class inherited QSortFilterProxyModel and as the documentation states methods like data are virtual, thus I can override them. Sadly, I wasn’t near anywhere that.

I know the proxy model was working. Why? It rendered the rows equally to the model’s rowCount. So, I’m baffled indeed. Whatever it inherited it certainly wasn’t QSortFilterProxyModel, my guess is that it called directly the abstract method…

But why data… god knows. I know I tried to override a few methods, and to my surprise I couldn’t get anything working with index, and parent.

Hopefully tomorrow I’ll have better results…

sweet baby jesus, fixing typo

Another take at const-correctness and “optimizing” code.

Most of what I did today was cleaning code… or refactoring code. I wasn’t happy with the current codebase as in there were a lot of copies of strings being made instead of simply making references and that’s what made it so unsatisfying as I was duplicating codes of data.

One of the aspects of my application is that most of the time I don’t need to modify data thus using const & comes handy most of the time plus it helps the compiler as much as it helps me if I make a wrong assignment if(cookies = cream). It took me like 2 hours to fully refactor the codebase … I wouldn’t even call it refactoring at this stage as nothing needed to be moved away.

After I was done with the task I sat down, scratching my head wondering why do I have so many unneeded objects in the heap. Well, the wonders of both C and C++ is having the ability to write your allocators and how you literally manage memory. Objects in the heap–mostly created by new–can be harder to keep track and free in a large codebase, thankfully shared pointers, scoped pointers have been introduced, and yet, people like me still use naked pointers–sorry, at least give me points for doing my homework.

Interestingly while I’d dare say with my naked eye I’ll say that the app launches faster. However Qt isn’t that forgiving as having QML Engine running can cost you 40-50mb of ram, just starting. It’s actually a tiny price to pay compared to what you get with Qt in exchange. I’m sure I could lower the memory usage but… who knows…

Either way… I must say I’m proud of myself, with the progress I have done so far. To the mere veteran, or armchair programmer it may not mean much; but to me means a lot as I have stayed truth to my goal of learning C++.

The fight isn’t over yet

So… after writing Onward, I say! I spent pretty much all day looking for answers. And I found them. Turns out how I was interacting with the database was sort of wrong, I need to tweak some things:

ta-da!

I was able to maintain a pretty responsive UI while the internals worked as hard as it can to make data available. Then I ran into a threading issue where quite frankly I had to rewrite the parser two times. Most of the rewrites were because I failed to understand how the stream parser worked rather than the source being faulty. Turns out that the threading issue was caused by a while never ending loop due to the parser never moving to the next element so it always gave me the same one.

All in all. I managed to solve the database issue, refactored the mechanism that gets the data and it’s oh so much better now; I managed to remove the complexity of it and now I just do a few conditional checks to see if everything’s okay.

The new parser seems pretty stable, rather… it’s probably the most stable parser I have written… so hopefully I won’t have to do any serious changes.

I think I can finally take a breather…

Onward, I say!

Writing this as a relief and to vent a bit… I guess.

So first of all Happy New Year! Hopefully this year will be filled with pure awesomeness. Secondly, my god managed network requests can really burn you.

I stumbled upon two serious bugs, one being that Atom reader wasn’t doing its job properly and I ended up writing it completely thus now it shouldn’t have any problems. Now, parsing through content can be sometimes tedious but manageable. I have done it so many times in the past in unrelated projects that I more or less know how to handle certain issues.

Networking sadly… is a new issue for me, but see my conclusion wasn’t that the tool was bad… rather that my application’s design is not bad but doesn’t really adjust to how QNetworkAccessManager plays. The designed classes to read Atom and make requests were both really bad designs as in, at least in the network part, was naively crafted. I had this great idea of stress testing my application to see how it would handle large amount of requests, in turn I had to implement around 300 line of codes to fix a few annoying issues and thus took care of:

  • Web servers don’t play nicely, don’t expect it to accept your request if your request don’t have headers like Accept, Accept-Encoding, User-Agent. I then by mistake added gzip in my Accept-Encoding which forced me to remove it as I currently have no interest in reading compressed content (yet).
  • No timeout. No timeout is just that. If your request doesn’t make it don’t expect the reply to erase itself. I didn’t have any timeout management
  • No supervision on how many requests to make per… … per what? there wasn’t any restriction thus it made the QNetworkAccessManager behave completely different.

The solution I have created felt so hack-ish I’m not slightly happy about it thus I’ll have to redesign how the network part interacts and how the network part interacts with the database.

While I was testing I ran into another problem: there’s no way to prioritize signals in Qt so I have to do the abstraction myself, which I don’t mind. To me providing a responsive interface is more important than the availability of data (which should fall into a “normal” priority). So in that department … I literally have to refactor a large portion of the application which is scary itself.

But… I must cut my losses and carry on. Most of the problems have been bad design itself… which is good and bad. Good because most of the logic is sound, bad because the design is a clusterfuck to manage.

Aight, well… go me, I guess.

The fruitful summer of 2015 to now

Screenshot from 2015-12-29 11:43:42

Basically the graph here has been my progress so far. Back in June and July I conceived the idea of creating an application. In August I made my first commits using Qt Widgets but as it stood it wasn’t easy to provide animations and transitions with widgets so I contemplated about using QML/QtQuick, to tackle two different languages took a toll on me I had to take breaks for a few days. I barely knew C++, heck, I still don’t know much about it but most of its syntax I can recognize and digest compared to when I started. And QML is sort of like a this weird technology that is so implicit in nature it drives you insane, at the same time the more you use QML the more you realize that your whole application can react to practically anything, it’s like using Qt’s connect on steroids, seriously.

And that’s what so scary about QML in a way. When I started writing QML I always overdid my QML types to the point I had to refactor or rewrite it again. Now in December I use less built-in types and do more in a way because I’m aware of QML’s behavior and how most of the anchors/layouts lays/positions itself.

If you notice, there aren’t almost any commits in the month of November I’d say I was taking care of personal issues and dealing with college that kept me away from programming. I was also really burnt out due to dealing with two languages so there’s that.

Well, I hope I can start this January of 2016 with a bang finally releasing my app.

Synchronize everything!

I think today is the first time I’ve felt that I’m halfway done with my application. It’s a very intoxicating feeling and at the same time there’s a lot to be done in the user interface area, like really a lot to be done.

I spent most of the day fixing a few icky issues with the model I’m using because as of now I’m using it for general purposes since I didn’t want to create another model that basically does the same thing, the exception happens if I need to work with a very specialized visual that needs its own model. Thankfully I was able to cram all the necessary roles in the model and I don’t have to maintain to different models because while there aren’t much to models… they can get slightly confusing to work with.

Each day that passes and I make progress there are times that makes me wonder if this is all worth it, for all I know my app could be a dud. There’s also the experience gained from doing it… well, it’s all uncertain I guess no use thinking about it.

Either way, I may end up refactoring and tearing down a lot of useless and questionable design decisions let’s see how it goes.

6 months ago I started this personal project

It feels like forever.

Even if it’s all I talk about in this blog I have I decided to keep going with this as if I kept pondering whether to do it or not I would have never been at this stage.

2015-12-25

I refactored and added plenty of code this week. Refactoring because let’s be honest when I’m faced with a new framework and a new language I’m bound to code little building blocks of what we call “experiments”, the very nature of not knowing pushes you to make spaghetti code in some aspect but you must be always willing to part ways with your old code and refactor as much as you can.

View post on imgur.com

Yes, yes. Give into the “Yes!” chants. (Daniel Bryan if in case you aren’t into the whole wrestling thing).

After refactoring, implementing new code, and deleting old code it feels oh so much better than before to the point that I feel pretty satisfied of my work so far in the app. I think what I’ve learned throughout the whole journey of this will be invaluable in the future if you consider the investment in C++ and Qt worth it that is, which to me it is.

With the changes I’ve done I can finally concentrate on the interactions between the database backend and the models involved so it’s always synchronized.

In a related note, I must say I’m impressed by the blazing speed of SQLite/Qt in general. I still have a few worries about it but… so far it’s been a pretty darn straightforward easy to use abstraction.

Thing is I’m doing my best to keep a good pace and finally finish it. I hope all my efforts eventually pays off, if not… I won’t really call it a failure as I gained so much in the process.

Time will tell..

A tiny bite of GTK3

Update: I was able to fix YCM. Pretty happy about that 🙂

What a day it has been for me. I’m totally wasted after pretty much driving all day, I had no real plans for today either way but I wanted to get a bit of a head start with GTK3.

I’m a vim user, wait, I was a vim user. I’m not sure if it’s my disdain of spending too much time trying to get things to work but as of late Vim with YouCompleteMe has been completely crap, no iostream, no gtk, no sdl, nothing, nada. I’ve been left completely in the dark by YouCompleteMe. After some basic debugging and working with paths and realizing that compile_commands.json wasn’t making the cut as YCM wouldn’t bother checking ALL the methods available; I gave up, really. Opened CLion and with the same, unique header of #include <gtk/gtk.h> I was able to get all the methods, even the deprecated ones.

I hacked CMakeLists.txt to support c and find the right GTK3 headers and in less than 5 minutes I was already writing C with GTK3.

Screenshot from 2015-12-20 10:29:57

I’m honestly not sure how to feel about it. What I mean is that GTK3 design feels … weird in almost every account compared to Qt4 or Qt5. I’m more familiar with how things are laid out in Qt and this isn’t because I’ve been developing an application with Qt but because I’ve used other frameworks with a similar per module design. The classes makes sense, there’s nothing funky going out you also don’t have to write gtk_label_new() followed with a gtk_label_set_text() (yes, there’s gtk_label_new_with_text()) the whole “zen” is gone.

QLabel label("Hello");
label.setText("Hello there");

I’m biased as I like working with Qt, and not only that I’m probably being unfair as GTK3 design is meant to be different as it’s targeted (mainly) for C, and doesn’t support an object oriented approach unless you use gtkmm, which I could I guess.

This is just playing with GTK though, honestly I think I would go with Qt in the end. I WOULD like to write a minor app with Gtk3 some time in the future though.

Lesson learned: Always use layouts, _always use layouts_

Ever since I got myself started using QML I have run into some funny business dealing with certain margin sizes. One of my oh so bad ideas was to recalculate the margin of an Item that’s already in a layout totally disregarding the Layout.margins property that’s available in every Layout container. This sounds all simple, seriously, it feels like one of those things that you wouldn’t expect to do but to those who are still digesting QML it’s a very big miss.

Anyways, always use layouts and its available properties or you’ll be in a world of pain.