“Taking a break”, I wish!

There’s a lot I want to talk about over this month. I think I have enough time to do so, but today I want to talk about letting go projects. Or rather, focusing on delivering. I was supposed to deliver my first app a month ago. I couldn’t due to a lot of complications with Qt/QML in general. TextAreas with images would go up to 200-300mb in memory.

It was insane nonetheless.

I applied workarounds that were quite fragile… in the sense that “I’m doing something I know I shouldn’t because this technology should support it”. I mean, who would have thought parsing and traversing the DOM in C++ would be a giant PITA? I didn’t. A lot of setbacks were because I kept pushing on with the pretense that it was all well worth it since it was C++. I mean, yes, there was a certain leverage using a low-level language. However, does it even matter? I mean, with QML you can easily reach 60-80mb without doing anything.

On the bright side, I never gave up trying to deliver the app.

The last few days I have been researching on Universal Windows Platform (UWP), UWP JavaScript bindings, WinRT Components, integration with Windows Phones. I must say that I am extremely impressed with Microsoft’s work. I’ve read plenty of developers mixed feelings about UWP, heck, even I found the whole thing confusing. UWP, WinRT, metro, whatever buzz name they’d come up.

2016-07-04 (2)

2016-07-04 (1)

As you see in the images. I’ve been cooking up something, something that I really hope to release eventually. When I jumped to C++ I didn’t know anything about it. I didn’t know much about C++, or about Qt for that matter. With UWP, a lot of things comes “naturally”. The permissive nature of UWP with JavaScript bindings is insane. I thought, “wow, QML is awesome!” without ever batting an eye to UWP. When I used UWP JS engine, it was of course a browser. I think at best I used like 20mb of memory. It even ran embedded flash…. something that QML is quite incapable at the moment.

UWP reference (mostly WinRT API reference) feels pretty solid so far. There are times where an implementation won’t work, for example, grabbing my profile picture from the OS. But, in a way this is why WinRT components exists. You can create components in VB, C++, or C#; these components are the bridge for anything that requires intensive processing. Let’s say, “video processing” or “audio processing”.

After spending a whole day poking UWP I feel like this is the right direction. The only thing is that I won’t be able to target Android, sadly. I could fix that with Apache Cordova… but, for now I’m just gonna target Windows devices.

Toying with VS Emulator, Hyper-V + Linux, Apache Cordova, and Windows 10 Virtual Desktops

You know this weekend I took my time to explore a few things in Windows 10 and Visual Studio 2015. For starters, every time I installed Windows, being a Windows Pro version owner I never bothered checking the features it offered, such as BitLocker.

Hyper-V

So, the thing about Hyper-V is that it’s something really, really abstract to the Windows user. Unless you poke the bear you won’t really know what’s in there. Hyper-V is a solution similar to Virtual Box, VMWare, it provides virtualization capabilities to your Windows. Now, why I mention Hyper-V is because it was almost a routine for me to always install Virtual Box after installing Windows, then it occurs to me that we already had Hyper-V to begin with.

I loaded up OpenSUSE Leap ISO and to my surprise everything worked flawlessly.

2016-05-15 (1)

2016-05-15

2016-05-15 (2)

You still need to configure the network through the Virtual Switch Manager in Hyper-V else the network won’t work, it only takes a couple of clicks.

Virtual Desktops

I’m a big fan of virtual desktops, every time I winded up using Windows part of me wanted to use virtual desktops because it was a way for me to organize my applications through tasks. Windows 10 finally, after many years, went ahead and added virtual desktops to the mix.

2016-05-14 (2)

2016-05-14 (1)

2016-05-14

Apache Cordova

In my internship I worked with Angular JS extensively to the point that I became interested in making mobile applications using HTML/JavaScript. I know of the existence of Cordova, however I’ve never used it mostly due to no interest in it. Curiosity took the best of me, I launched Visual Studio 2015, installed the required components and within an hour I started my little adventure on playing with Cordova.

I have mixed feelings about it. At first sight it feels like the application structure will crumble apart in any moment. It’s not the same having a set of widgets to work with, as in there’s: ListView, WebView, TableView, Layout controls, Buttons, Combo Boxes, and so much more available programmatically through an API. With Cordova you have HTML5/JavaScript and pretty much 9000000 javascript scripts available. Does it mean it’s “stable”? If you stick to jQuery, AngularJS, etc then sure.

Either way, Cordova is new to me and things like debugging, poking around seems too loose at the moment.

Visual Studio Emulator

I have a beef with this emulator… I can’t deploy Qt applications with it as the shader program is not linked. I want to deploy Qt applications because else I would have to disable Hyper-V and I don’t want to do that because it requires me to reboot every time.

Overall my experience with VSE has been nice, it’s straightforward, I just wish it played with Qt nicely. The best part? Qt Creator can automatically detect VS Emulator emulators running so you don’t have to do any extra work.

What’s next in programming?

So basically I’m at my wits end. I haven’t had the time to finish my application. There are two things that have demotivated me greatly. First, the overhead of QML is too much to the point that I might have to drop it. This means I’ll have to come up with a way to build a new interface out of html/js/websockets.

I’m up for the challenge, I really am, yet I feel awful about it because I’m scrapping a large portion of the presentation for something that takes time to do. I don’t think the Qt team can optimize the QML Engine in less than two months. The original schedule was to release at March, yet due to circumstances (doing internship, loss of motivation due to the fact that I have to scrap code) I haven’t sat down to work on the app.

I think at worst this is the part I’m just banging my head against the wall since I don’t want to lose my work in vain.

Goodbye, QML

I’m “abandoning ship” and hopping on Xamarin/Microsoft bandwagon on creating mobile apps with C#. It pains me to say that getting things to work requires a lot of effort with QML. I feel like there’s so many, many things it can do yet it tries to be so many things. At the end of the day, I was put against these options:

  • To deliver applications rapidly and gush out iterations as I go
  • The need of high-level language as C++ can be a little tedious to do some things. One can easily run into type issues.
  • Less time working on C++ backend, less time trying to understand why certain QML elements consumes over 800MB in memory (hello TextArea, you sure can’t handle 10 high-res images).

Overall, QML and C++ combo, as powerful as it is has too many memory usage issues, a steep learning curve.

Today I went over Xamarin stuff, it was relative easy to get started. I noticed that I could finally work with Activities, unlike Qt, which you have to spend a lot of time learning the android/qt cycle.

It pains me, but as most of you I also want to put my application out there in the wild as “soon as possible”. With Qt/QML I completed one mission, and that was learning C++. I did, maybe not to an expert level, but I grasped plenty.

Updates, updates everywhere!

I’ve been watching Microsoft’s Build event, been doing so since last year and this year has been pretty darn exciting. I’ve yet to dabble into C# mostly due to most of my energy going to C++, it’s apparent that I want to work with C#, after working with Qt I feel like the reach for “easy mobile development” is a mere pipe dream. I feel… disappointed? I guess. The truth is that I’ve spent my time doing more workarounds and hack-ish solutions that actually working on my application. QML is a very, very insanely powerful language, the things it brings to the table is just awesome. Yet, the price to pay is having poor documentation on many QML elements.

Putting my feelings aside on Qt/QML for a moment. I think that I will eventually push for C#, in all honesty I just want to stay with one, maybe three languages to work with. Right now I’m just a jack of all trades, and I don’t like it that much.

Anyway! More updates

Recently I bought an SSD, a 256GB drive and it has been a huge step for me. I usually had to wait 5-9 minutes for a full boot in a normal HDD. Now it boots completely in 20-30 seconds.

2016-03-29

I also bought off a 970 GTX from a friend. I’m way behind games, and well, hardware in general. Once I installed it on my PC it blew my mind after playing games like Witcher 2 in pure 60FPS, Call of Duty: Ghosts was the one that called my attention the most with insane level of detail. I plan to snatch Advanced Warfare eventually too.

2016-03-26

Both hardware pieces have contributed greatly on my PC. The boot time is no longer a problem and Visual Studio opens in less than 10 second. Meanwhile games I had problem playing due to framerate issues are no longer an issue.

Now… there’s only one big worry I have. Software development interviews and getting hired. These are possibly my biggest challenge, as a person new to the industry I just don’t know what to expect.

Reducing complexity

I feel awful, yet happy, yet tired.

A bit of an annoyance is when you spend hours doing something, then a week or two later you see a new library that does that and simplifies the code greatly; allowing maintenance and readability at the same time.

So, take it as you may but if you are using classes like QXmlStreamReader or QDomDocument do yourself a favor and try pugixml library. It’s said to be faster than the previously mentioned. In terms of code readability QDomDocument is the nearest that matches pugixml due to its DOM api, yet it’s also slower and consumes a hefty amount of memory in exchange, where as pugixml stays lean.

The inadequacy of HTML, delving into C, and pondering remote jobs

I’ve been working with HTML in low-level languages like C/C++ and I must say that it’s been a rather frustrating experience. One would think that the task I’m doing should have been done ages ago, since the conception of the project. Alas, that’s not quite right. What you fail to realize is that you should NEVER underestimate the dirtiness of HTML outputs all around the world. This whole week has been about cleaning/scrubbing/sanitizing HTML. I ended up grabbing to rather awesome third party libraries, one called pugixml and the other libtidy (also known as tidy-html5).

libtidy, being a pure C library took me a while to get the hang of as I’m not that experienced in C. Even though I’m writing C++ it doesn’t mean I’m working with pointers the majority of time (mostly on the stack). It was more of a game of “copy this chunk of memory and put it as an argument in this C++-powered method so I have more control over it with the classes I’ve built. But I needed it to work, regardless on how long it took to get right because without tidy it would be really, really hard to do any scrubbing. pugixml and other XML parsers can get quite cranky while parsing misplaced tags.

Which brings me to pugixml!

Apparently QtXml module is no longer maintained as it has reached a matured state. The array of classes QDomDocument have in general feels suiting to do the job, sadly the documentation stresses that it uses too much memory, and apparently “pugixml is faster than QXmlStreamReader”

In the end. I chose pugixml because it’s just simple, it doesn’t have the annoyances QXmlStreamReader brings.

Another subject I wanted to bring is the interest of the C language.

I’ve been thinking of learning more about C. At least the basics to defend myself from cases like libtidy. I also wanted to learn more about C due to Gtk3 and due to getting closer to system programming (not that C++ isn’t capable). Mix this interest with me wanting to work with microcomputers or embedded systems and I might make something out of it.

Lastly there’s that thought of me wanting to get a job related to my field. I might try to get a job remotely, well, time will tell. For now I’m too tired, mentally spent, yet as I finish this post I have a class to attend to in college. ­čÖü

SQLite and the importance of indexes

I didn’t spend too much time on this issue, but as someone who knows the basics to intermediate in SQL it still caught me off-guard on how easy it is to have insanely slow query performances.

Take an innocent query like this. Once would think it has some sort of meta information stored already, but it doesn’t

select mytable.*, 
   (select count(*) 
    from mytable2 
    where mytable2.id=mytable.id ) as count
 from some_table;

What it does is that it will get all the columns in mytable and additionally it performs a COUNT based on id in a subquery thus the result is the total count per ID.

It took 7662ms to perform the query on two tables that have 387, and 4,800+ rows respectively. That’s not acceptable in any way, putting that sort of query in production will make your customers leave eventually due to the slowness of the system.

I googled a bit on the issue and found that sqlite doesn’t store any meta or index. I went ahead and create an index based on the second table, as the first one already has an index (primary key).

I managed to trim those seconds to 240ms, which imho is pretty acceptable, yet makes me feel uneasy.

The result?

select mytable.*, 
   (select count(ROWID)
    from mytable2 
    indexed by my_awesome_index
    where mytable2.id=mytable.id ) as count
 from some_table;

That’s all it took.

I’m still not completely satisfied with results like this, I will continue finding a way to trim those 200ms away to 20 or 50. I have an idea on how to do it but for now this will do.

Aftermath of memory management / designing for mobile / the need of something better

The aftermath… of all my chasing is that I got nothing. You heard me right. I spent days figuring out what was wrong just to receive one hell of a slap to the face that the OS doesn’t necessarily claim the memory back. But, I also noticed that memory started to be freed after changing the parent of QNetworkReply to 0, meaning I manage it manually.

Regardless, after some testing I’m happy to say that while I messed up big time the application is working as expected. I think the lowest I saw was the app using 18mb and that for me is more than enough.


Now onto designing mobile applications. See, I’m not a designer I know that, you know that. I don’t think I will get this right… but it’s time to let go honestly; I have spent too much time trying to have the “perfect” design and such thing doesn’t exists. Now I will sit one more time to work on a new mock up.


Perhaps I’m losing my mind. The more I think about it the more I notice that I’ve been preventing myself from experiencing something good. Not so long ago I pondered on getting a Macbook Pro for various reasons. The first being development, the second being Unix, the third being a polished desktop, the fourth being an stable desktop.

I get it, you love linux and I do too. It’s not enough. I want better printer support, better wireless support, better battery life because Linux can’t even do battery life management correctly and for a battery that last 2-3 hours using Windows I get only 1 hour. I just want a better desktop experience. See, windows provide that without me having to do much. Linux… provides that to a certain degree, once things start breaking apart due to upgrades then good luck because you will need it depending on the issue. The whole “well, you can just reinstall it back’ isn’t even a solution but a living proof that linux is still behind desktop experience.

It feels awful though, being a supporter and all, but once you start luring people in and seeing them running into problems it just feels all kinds of wrong.

Fun with Python3 and PyQT

I guess I should start by saying that I’m using PyQt4 mostly because… well, I don’t have an excuse, just pure laziness when I started writing the code. This is mostly an exercise where I toy with Python and Qt in general; I find it “relaxing” to work on other stuff than eating C++ every day, and Python being as delightful as it is I know I don’t have to worry about memory management at all.

Screenshot from 2016-01-28 23:25:06

So, in this little app I just wanted to play with slots/signals to see how it was done in PyQt. The documentation in PyQ is messy and that is an understatement, I had to poke a lot of pages in the documentation just to see how it was done. I’m not sure if PySide is any better …

Anyway here’s the code:

from PyQt4.QtGui import QSpacerItem, QApplication, QTextEdit, QPushButton, QWidget, QLabel, QVBoxLayout, QMainWindow
from PyQt4.QtCore import Qt, SIGNAL, pyqtSignal, pyqtSlot, QObject
import sys


app = QApplication(sys.argv)

window = QMainWindow()
window.setWindowTitle("Qt Rocks!")
window.setMinimumWidth(500)
window.setMinimumHeight(500)

widget = QWidget()
textEdit = QTextEdit(widget)
textEdit.setFixedHeight(50)
pushy = QPushButton("Say my name", widget)
spacer = QWidget()
label = QLabel("Type your name",widget)

@pyqtSlot()
def receiver():
    label.setText("Hello, %s" % textEdit.toPlainText())
    return

pushy.clicked.connect(receiver)
# _clicked_ is a signal from QPushButton. This form of connection doesn't 
# get detected by YouCompleteMe in the autocomplete... so gotta read up documentation

layout = QVBoxLayout()
layout.setAlignment(Qt.AlignHCenter)
layout.addWidget(label)
layout.addWidget(textEdit)
layout.addWidget(pushy)
layout.addWidget(spacer)
widget.setLayout(layout)
window.setCentralWidget(widget)
window.show()

sys.exit(app.exec_())

I find that if I’m going to invest a lot of my time with Qt. I think Qt/Python makes a devastating (awesome) team in the sense that whilst you don’t have to worry about memory management you can fully focus on writing and gushing out that vital code the business needs . I’d say that depending on the nature of the application Python/Qt will do on most cases. These days memory is cheap, hardware is more powerful than ever… nevertheless always know the nature of the app before choosing a language.

I concluded that hopefully in my next project I’ll be using that combination.