Aftermath of memory management / designing for mobile / the need of something better

The aftermath… of all my chasing is that I got nothing. You heard me right. I spent days figuring out what was wrong just to receive one hell of a slap to the face that the OS doesn’t necessarily claim the memory back. But, I also noticed that memory started to be freed after changing the parent of QNetworkReply to 0, meaning I manage it manually.

Regardless, after some testing I’m happy to say that while I messed up big time the application is working as expected. I think the lowest I saw was the app using 18mb and that for me is more than enough.


Now onto designing mobile applications. See, I’m not a designer I know that, you know that. I don’t think I will get this right… but it’s time to let go honestly; I have spent too much time trying to have the “perfect” design and such thing doesn’t exists. Now I will sit one more time to work on a new mock up.


Perhaps I’m losing my mind. The more I think about it the more I notice that I’ve been preventing myself from experiencing something good. Not so long ago I pondered on getting a Macbook Pro for various reasons. The first being development, the second being Unix, the third being a polished desktop, the fourth being an stable desktop.

I get it, you love linux and I do too. It’s not enough. I want better printer support, better wireless support, better battery life because Linux can’t even do battery life management correctly and for a battery that last 2-3 hours using Windows I get only 1 hour. I just want a better desktop experience. See, windows provide that without me having to do much. Linux… provides that to a certain degree, once things start breaking apart due to upgrades then good luck because you will need it depending on the issue. The whole “well, you can just reinstall it back’ isn’t even a solution but a living proof that linux is still behind desktop experience.

It feels awful though, being a supporter and all, but once you start luring people in and seeing them running into problems it just feels all kinds of wrong.

Fun with Python3 and PyQT

I guess I should start by saying that I’m using PyQt4 mostly because… well, I don’t have an excuse, just pure laziness when I started writing the code. This is mostly an exercise where I toy with Python and Qt in general; I find it “relaxing” to work on other stuff than eating C++ every day, and Python being as delightful as it is I know I don’t have to worry about memory management at all.

Screenshot from 2016-01-28 23:25:06

So, in this little app I just wanted to play with slots/signals to see how it was done in PyQt. The documentation in PyQ is messy and that is an understatement, I had to poke a lot of pages in the documentation just to see how it was done. I’m not sure if PySide is any better …

Anyway here’s the code:

from PyQt4.QtGui import QSpacerItem, QApplication, QTextEdit, QPushButton, QWidget, QLabel, QVBoxLayout, QMainWindow
from PyQt4.QtCore import Qt, SIGNAL, pyqtSignal, pyqtSlot, QObject
import sys


app = QApplication(sys.argv)

window = QMainWindow()
window.setWindowTitle("Qt Rocks!")
window.setMinimumWidth(500)
window.setMinimumHeight(500)

widget = QWidget()
textEdit = QTextEdit(widget)
textEdit.setFixedHeight(50)
pushy = QPushButton("Say my name", widget)
spacer = QWidget()
label = QLabel("Type your name",widget)

@pyqtSlot()
def receiver():
    label.setText("Hello, %s" % textEdit.toPlainText())
    return

pushy.clicked.connect(receiver)
# _clicked_ is a signal from QPushButton. This form of connection doesn't 
# get detected by YouCompleteMe in the autocomplete... so gotta read up documentation

layout = QVBoxLayout()
layout.setAlignment(Qt.AlignHCenter)
layout.addWidget(label)
layout.addWidget(textEdit)
layout.addWidget(pushy)
layout.addWidget(spacer)
widget.setLayout(layout)
window.setCentralWidget(widget)
window.show()

sys.exit(app.exec_())

I find that if I’m going to invest a lot of my time with Qt. I think Qt/Python makes a devastating (awesome) team in the sense that whilst you don’t have to worry about memory management you can fully focus on writing and gushing out that vital code the business needs . I’d say that depending on the nature of the application Python/Qt will do on most cases. These days memory is cheap, hardware is more powerful than ever… nevertheless always know the nature of the app before choosing a language.

I concluded that hopefully in my next project I’ll be using that combination.

Back to business

So I’m finally back from my one week break. I’m somewhat disappointed with my lack of exercising these days, and much of it boils down to my sciatica interfering when I start working out. That aside, it’s been a roller coaster of emotions.

screenshot
2016-01-26-232237_1600x900_scrot

For starters I moved my laptop to use elementaryOS, and so far I have no regrets. It’s been a delightful experience. I think the next iteration of it is around the corner (maybe?) with the next Ubuntu LTS and it’s going to be a hell of an upgrade. What I worry at the moment is the lack of updates in Spotify I hope that when Ubuntu LTS kicks in they finally update their client to use a recent version of libgcrypt.

As for my application. I’ve been pondering for a while what to do with the user interface as it’s been a very stressful task of bringing something good to the table. My fault lies within the need “to be original”… sadly original these days is incredibly hard so I’m going to focus on bringing a sane and solid experience even if the design in the end reeks of “boring”. It really stresses me as while part of me want to provide something really solid I can’t come up with a good interface without sacrificing something. Well, tomorrow I’ll be sitting once again sketching a new and predictable design.

I’m no designer though… so all I can do is just try.

Looking back: OpenSUSE 42.1 with latest KDE packages

I’ve been killing some time with other Linux distributions. As far as I can tell one of my favorite Desktop Environments is still struggling out there. plasmashell crashes everywhere, right now the right side of the panel is a mess with icons overlapping on each other. It makes me a sad penguin!

I think… in 2015 I just stopped caring about the current progress of KDE… as much as it hurts to say. Amusingly–for some–I’m looking forward to see the next iteration of elementary OS or maybe Linux Mint Cinnamon. The next iteration of Ubuntu LTS is around the corner needless to say, so I’m guessing that both elementary OS and Linux Mint will be ready when the time comes.

I am looking forward to that at least. I am glad that I chose to test over in a virtual box before installing this on my laptop, else I would have been pretty upset.

As for OpenSUSE 42.1 (Leap) I got no complains whatsoever. It seems relatively stable, excluding the KDE problem as that’s a problem that lies with upstream, not OpenSUSE team. I got Spotify working in less than 10 minutes, sufficed to say I’m impressed on that since last time I had to hop through a lot of obstacles just to get it working.

Well, I hope 2016 is a fruitful one for Linux users. I also hope that KDE finally stabilizes this year.

Chasing the butterflies in GDB

You know when you are chasing bugs it can take a lot of time and energy. While I’m still fatigued I ran into a small design problem, a decision I wrote without really thinking how QNetworkReply would react thus I ran into a lot of crashes. See, crashes you at least have info to work with as the GDB would stop and deliver the frames that you will later check.

When it comes to CPU hog though, everything becomes a chase. Sometimes you either require gut feeling, or a lot of poking through the threads to see if a thread is stuck in a loop. Sadly, due to my limited experience with GDB I just can’t say “hey, alert me if you are busting a nut over there”.

Before I go on, the reason I ran into this specific problem was because I’m taking advantage of QObject parent/child relation. At the same time I’ve been moving away from pointers or anything heap related unless I really require them, which so far I haven’t had the need. This also means less memory management for me. So, if a QObject that is a parent dies, it takes the children along with it, which is incredibly ideal for me.

See, these butterflies I’m talking about aren’t real. I made the mistake of doing a test with data that is 10 years old, with address that I have to poll with QNAM. There’s a lot of redirection/error handling that needs to be in place–which I have taken care of–so I noticed that a thread (apparently a HTTP Thread by NAM) was hogging a core for itself, then after 30 seconds to a minute it would stop/terminate. My first thought was to poke the parsers, since last time I ran into a loop problem as they weren’t able to handle a instruction so it went into a loop. I think, maybe some wouldn’t call it CPU hog as to be honest said CPU hog stopped.

I got fed up; it was a dead end. If there’s a problem with my source code, I can’t detect it. I’m flushing QNetworkReplies as safely as I can, aborting them as safely as I can. So… time to go with a fresh test case. I’m using recent URLs of updated sites and have noticed that there’s no CPU hog anymore. Granted, I still have to keep testing.

I feel like I wasted a day though. A dead end with no clue. It’s a bit frustrating as all these speculations of a memory/cpu hog problem wasn’t real, or maybe it is and I haven’t really scratched the surface of it. On the bright side I cleaned up an class and its behavior is more predictable than before.

On a side note, I feel like I’m doing premature optimization and I should really stop. There has to be a legitimate issue before I jump into these types of searches.

Easy steps to move away from GMail

2016-01-12 (1)

So if you are like me, paranoid and all then by now you have been wondering “how to move away from gmail”. There are e-mail services like Fastmail (check my sidebar) that offers to import from gmail, but it can only be done once else you might fall into duplication hell. Today I’m going to go over what I did:

Starting fresh

Wait, what? You deleted all your e-mails? No, of course not. I have a decade worth of e-mail and I can’t be bothered to go through all that data. First step is to go to Google Takeout and back up your data. Back up your mail data and download the archive when it’s done.

I did want to start fresh. There’s a catch on why though, I don’t need to keep track of my e-mails. There are people that do have to keep track of them maybe due to work.

Use a encryption tools like Veracrypt, Bitlocker, etc.

2016-01-12 (1)

While Bitlocker gets a bad rap for not being open source thus not being able to audit [insert childish “b-but it’s Micro$soft” comment]. What I’ve read from it personally I consider it safe as long as you don’t back up your recovery keys to Microsoft. If you have invested yourself in Windows please do yourself a favor and research more about Bitlocker, else you could also completely encrypt your physical volume with Veracrypt.

The point of using these tools is to create a container, size may vary due to your mbox file. I will continue explaining later in this post.

Choose your e-mail client

Personally there are many ways to access your mbox file. Heck if you wanted you could create a virtual pc (virtual box) with linux, all encrypted with your mbox file in it and read it through Thunderbird. So it would go like this (virtual image fixed size gets to a Veracrypt container, load the image whenever you want with Virtual Box).

That’s too complex, seriously. What I did was simple, I chose/downloaded Thunderbird from Portable-Apps and installed it in my C:/ to later move it to my Veracrypt container.

Install ImportExport Tools for Thunderbird

Installing portable Thunderbird was the first step. Now install the ImportExport tools because with that tool you can import your mbox file to Thunderbird. Note that this process while take minutes to complete if the mbox is too big.

Let it index

This also may take minutes or hours. Once you have finished importing your mbox, let Thunderbird finish indexing.

Calculate the size of your Thunderbird portable folder

2016-01-12 (2)

Once it has finished indexing, go to the installation folder of Thunderbird and calculate how big it is. Let’s assume that it’s 2GB, so go open up Veracrypt and create a container around 2.5GB or 3GB.

Move your thunderbird installation

Since this is a portable edition of Thunderbird you can just move the folder to the container. The point of this is that you can put your container in a USB and use it whenever you want.

Time to delete

2016-01-12 (4)

Please ensure that your container has all the data before deleting.

Delete all the data from your gmail account and the initial installation folder of thunderbird. We don’t want that data lying in your personal pc unencrypted do we?

Upload it to the cloud

You can upload it to Sync or Dropbox if you feel like. I prefer Sync because you can upload it and move it to the Vault that way it won’t be deleted by anyone.

Enjoy your privacy, and peace of mind. Congratulations your data is no longer on Gmail. Now with your data safeguarded on the cloud + encrypted, you can start using a fresh e-mail address. Do keep in mind that instead of using your real address, you can create aliases like if you sign up to Facebook or Pinterest you could say [alias]+[folder]@fastmail.com where as alias is linked to your real account, and +folder is where you want the e-mail to go.

So consider the possibility of a site getting hacked, and said site uses a weak hashing algorithm (heck, it could even be in plaintext). Using an alias will buy you time to change your e-mail password if it turns out that you used the same password in both accounts. The hackers will have to go through lengths to actually find your real account. Plus, they also can’t log in to your Paypal as the e-mail they got is an alias, taking in account that you also use different passwords. So yea, you get to buy time on all accounts. If you liked this, do consider Fastmail services check my sidebar.

The weird case of proxy model

Rest in peace, David Bowie

***

So basically I spent the day working with Qt’s proxy model. I didn’t get anything done, literally. I ran into a really weird problem, the proxy model I was working on wasn’t propagating methods like data. I added them both in the source code and header, nada, nothing! I’m baffled with this behavior, my class inherited QSortFilterProxyModel and as the documentation states methods like data are virtual, thus I can override them. Sadly, I wasn’t near anywhere that.

I know the proxy model was working. Why? It rendered the rows equally to the model’s rowCount. So, I’m baffled indeed. Whatever it inherited it certainly wasn’t QSortFilterProxyModel, my guess is that it called directly the abstract method…

But why data… god knows. I know I tried to override a few methods, and to my surprise I couldn’t get anything working with index, and parent.

Hopefully tomorrow I’ll have better results…

sweet baby jesus, fixing typo

The weird case of foreign languages

I have noticed over the months I have been working on my app that managing foreign languages like Japanese, Chinese, Korean, etc takes a memory hit to the point that it spikes up the memory usage–as one would expect. Sure, I expect more memory usage… but we are talking about a spike from 60mb (normal usage) to 80-100mb usage per entry–thus I was forced to call garbage collector manually as I couldn’t wait for the QML/JavaScript engine to do the clean up when there’s “inactivity”.

So, I was worried and added initial linux support to my app (never been planned to be released under linux) and found out that languages such as Japanese walks a fine line of 40mb memory usage. The garbage collector works twice as fast as well. For example, in Windows I’d call garbage collector and it doesn’t do it that fast. In linux? Blazing fast. It’s not much about the speedness of how fast it takes to free the memory, it’s the usage. I’m talking that Windows still takes 60-80mb+ (and up) while the linux build keeps walking the 40-50mb line.

Now, initially I suspected a memory leak, but that wasn’t the case. The Windows build just takes that much memory, and it worries me. Part of what keeps me at ease is that since this will be an android application; I can expect the same behavior that I get here in my linux mint in Android. Memory will be freed and the usage will be kept at minimum.

Granted: Pure naked eye seeing memory usage is not enough to suggest there’s a memory leak or that there’s a memory mismanagement, that’s the job of a profiler After running valgrind a few times I couldn’t find any memory leak just petty warnings, so I proceeded to guard against it, well, it wasn’t much about guarding as I honestly needed to free the objects that are no longer needed after X time. Plus, I also rationalized the usage that Windows is just being Windows… maybe Qt is pulling something that requires big allocations in Windows, but not in Linux for whatever reason. I do think that it has something to do with the font mechanism in general… but who knows at this point.

Spotting a minty taste

My family threw a party, but due to age, and humidity’s tag team partner, dust, it has decimated a lot of the music CDs over time. Plus my family track of music hasn’t been updated for decades so I was at my wits end since I refused to be exposed to more christmas music… seriously over the year it just…. gets unbearable, to know I have the potential to be grinch, huh.

Well, I came with the idea of connecting my Linux Mint powered laptop to the audio system but I didn’t have the cables to do so either. Luckily there was a stereo jack to 2RCA cable in my room–that I didn’t even know about–so basically the result became this:

mintyspot

Linux Mint, being a derivative of Ubuntu LTS release, I didn’t have to follow my own guide of fixing Spotify in Ubuntu Vivid… which makes me wonder if Spotify finally fixed their build. I’m a happy Spotify user, and I’d like to use the client in linux, not the weird web interface. Heck, you know, you might as well release the web client for linux.

I’ll elaborate:

I installed Fastmail’s Android application to access my mail weeks ago, and I noticed that their client is actually the web application embedded with a browser so all it does is API calls to the main servers and store whatever offline data it needs to.

It’s an option, only if providing the client to linux is hard for them. [avoid ranting about mostly clueless users who think cross-platforming is easy] (it’s almost a taboo to rant about users, it seems. Well, mostly counterproductive.)

In conclusion, Spotify is incredibly awesome. I’ve discovered a lot of artists thanks to it. Recently I have found that their Discover section is getting better and better, meaning we get to discover even more musicians that might be your cup of tea. There is a need for applications/services like Spotify for linux. Especially to premium users, we can sync our playlists and keep programming without a worry, or maybe sync music to your phone and go to exercise.

Another take at const-correctness and “optimizing” code.

Most of what I did today was cleaning code… or refactoring code. I wasn’t happy with the current codebase as in there were a lot of copies of strings being made instead of simply making references and that’s what made it so unsatisfying as I was duplicating codes of data.

One of the aspects of my application is that most of the time I don’t need to modify data thus using const & comes handy most of the time plus it helps the compiler as much as it helps me if I make a wrong assignment if(cookies = cream). It took me like 2 hours to fully refactor the codebase … I wouldn’t even call it refactoring at this stage as nothing needed to be moved away.

After I was done with the task I sat down, scratching my head wondering why do I have so many unneeded objects in the heap. Well, the wonders of both C and C++ is having the ability to write your allocators and how you literally manage memory. Objects in the heap–mostly created by new–can be harder to keep track and free in a large codebase, thankfully shared pointers, scoped pointers have been introduced, and yet, people like me still use naked pointers–sorry, at least give me points for doing my homework.

Interestingly while I’d dare say with my naked eye I’ll say that the app launches faster. However Qt isn’t that forgiving as having QML Engine running can cost you 40-50mb of ram, just starting. It’s actually a tiny price to pay compared to what you get with Qt in exchange. I’m sure I could lower the memory usage but… who knows…

Either way… I must say I’m proud of myself, with the progress I have done so far. To the mere veteran, or armchair programmer it may not mean much; but to me means a lot as I have stayed truth to my goal of learning C++.