Integrating libssh2 and boost::asio

When I first looked at boost::asio, I didn't quite get it. I looked at the tutorials and thought "That's too easy". Then, I looked at the examples, and tought "That's too hard". Then, as weeks went by, I found more guides and articles about it, and I hit the point where I could bring the porridge analogy home and say "That's just right".

On my pet project, I'm using libssh2. So, I thought "Why not use boost::asio to frame my usage of libbsh2? I could build my libssh2 C++ wrapper, based on asio". Since I'd had done some work on the ssh2_exec example, I chose it as my starting point for integration with asio.

I did do some googling first, to see if anyone had had a shot at anything like this, but found nothing I could use. So, according to Google, I seem to be the first misguid... er, I mean, brave soul to document my attempt at it.

Introducing tcp::socket was easy. And I got asio's portability, to boot. No more mucking about with #ifdef, WIN_32, WHERE_ARE_MY_SOCKS, and all that jazz.

Then, I took a closer look at libssh2's implementation. It's an API composed largely of what we might call "C's shot at coroutines". Each of these functions has a group of discrete steps, and maintains its state, so that it knows which step is currently executing. Since all this is non-blocking, the execution of a function is a loop of invocations until all its steps are done, at which point the state is reset, so that the cycle may begin anew, when that function is called again.

Regarding the "non-blocking" part - libssh2 implementation is always non-blocking. Its API may be blocking, if the user calls libssh2_session_set_blocking(LIBSSH2_SESSION *session, int blocking) with blocking != 0. This only affects the user's code (and only that particular LIBSSH2_SESSION). Once control is handed to the called libssh2 function, it is always non-blocking. libssh2 implements a macro loop, to ensure the behaviour is consistent with the user-requested blocking status - if blocking is on, the loop won't return until all the function's steps have been performed.

While going through their steps, libssh2 functions perform plenty of read/write on the socket. And all this activity will happen without any control being given back to the user's code, even if we're not blocking; in my case, this "user's code" includes asio. Which is also observing the socket. While at first this sounded troublesome, after further research, I'm inclined to believe it may actually be harmless. Let's see...

1. App creates instance of SSHWrapper (an originally - and brilliantly, if I do say so myself - named C++ wrapper for libssh2, using asio).
2. App invokes SSHWrapper.Connect(), which will open an ssh connection to a host, somewhere over the thing with the pot of gold.
3. SSHWrapper sets a handler on socket read/write (which calls, e.g., libssh2_session_handshake()), and starts an io_service event loop.
4. Activity on the socket fires our handler.
5. The handler calls the libssh2 function.
6. If libssh2 work isn't done, "rearm" the event handler before exiting and repeat steps 4-6.

We could have problems between steps 5 and 6; at this point, the libssh2 function could (meaning "most likely will") trigger socket activity outside of asio's control, which could create further read/write events. OTOH, at this point, a) there is no active handler for such activity (we will only have an active handler again on step 6; and b) all this work is happening on the same thread (my next few PoCs will cover multithreading, where I intend to find out exactly what "Asynchronous completion handlers will only be called from threads that are currently calling io_service::run()" means), so there's no risk of reentering the handler, even if it was still defined.

One detail was puzzling me, though. Looking closely at steps 3 and 4, you'll notice that there's activity on the socket before any libssh2 function is called, since the function is called only in step 5, in the handler itself. However, something was firing the events and calling the handlers. Some testing revealed that calling tcp::socket::async_write_some() with an instance of boost::asio::null_buffers() fires the event immediately, which does make sense - I wouldn't call a write function if I wasn't ready to write, right (I'm already conceiving some multithreading tests regarding this one, actually)?

So, even though some details are still a bit foggy, and merit further experimentation, I have a working example. This is equivalent to libssh2's ssh2_exec example, and I'll be discussing it here in the following posts.

Updating a Qt GUI from a boost::thread

OK, first things first.

After seeing how horrible the previous post looked, I've decided to look into the subject, to improve the readability here. After checking several alternatives, I've decided to use an online formatter to format the code and copy/paste the HTML into this blog. Some problems remain unsolved, but I'll leave that to another day.

Now, then. Lately, most of my work has been on PoCs (Proof-of-Concept), and I'll discuss one of those here, now: Updating a Qt GUI from a boost::thread.

For this, I used the following classes:

• WorkWindow. The GUI; inherits from QDialog.
• Task. The class that does the actual work and updates the GUI.
• TaskController. I use this to glue both classes together. The PoC doesn't require it, but a more real-world (i.e., complex) example would require something to fulfill this role.

WorkWindow contains a button that starts the task:

boost::function<void(std::string)> f_res =
        boost::bind(&WorkWindow::AddSearchResult, this, ui->treeWidget_1, _1);

    m_tc->StartTask(f_res);

We create a boost::function to call WorkWindow::AddSearchResult(), binding a particular widget to the call. This is part of what I called a few posts back a callback-with-a-sideline-in-information-hiding - whoever invokes this function object has no knowledge that one of the arguments of its invocation is actually a GUI widget. m_tc is a pointer to a TaskController, and this is the invoked method:

void TaskController::StartTask(boost::function<void(std::string)> f_res)
{
    Task t(f_res);
    m_thread = new boost::thread(t);
}

We simply create a new task and fire its thread. m_thread is a pointer to a boost::thread, held by TaskController. This is the task's operator(), invoked by boost::thread:

void Task::operator()()
{
    for(auto i : str_vec)
    {
        m_UpdateUI(i);
    }
}

str_vec is an std::vector<std::string>, and simulates the results of Task's processing. m_UpdateUI is a boost::function<void(std::string)>, and is initialized with f_res.

So, Task::operator() invokes WorkWindow::AddSearchResult(), passing a string:

void WorkWindow::AddSearchResult(QTreeWidget* w, std::string s)
{
    emit SigAddSearchResult(w, s);
}

As we can see, AddSearchResult() emits a signal, forwarding both arguments. And, on WorkWindow's constructor, we connected this signal with its slot:

connect(this, SIGNAL(SigAddSearchResult(QTreeWidget*, std::string)),
        SLOT(SlotAddSearchResult(QTreeWidget*, std::string)),
        Qt::QueuedConnection);

We use Qt::QueuedConnection to ensure the slot is executed in the GUI's thread, not on Task's thread. We could use the default value, Qt::AutoConnection, and Qt would choose the correct mode, by comparing native thread IDs.

Finally, the slot:

void WorkWindow::SlotAddSearchResult(QTreeWidget* w, std::string s)
{
    QTreeWidgetItem* elem = new QTreeWidgetItem();
    elem->setText(0, s.c_str());
    w->invisibleRootItem()->addChild(elem);
}

It's actually pretty simple. To recap:

- The GUI requests a task execution, handing the task a callback to update the GUI.

- The task is started in its own thread. When it has data, it invokes the callback.

- The callback emits a signal.

- The slot processes the signal, updating the UI. The slot must be processed in the GUI's thread, and this is enforced when both signal and slot are connected.

And that's it. As a final note, using QThread provides better integration, and simplifies all this somewhat, but the steps above remain unchanged.

Fake child window implementation in Qt

Well, the unorthodox idea didn't really pan out; the event handlers are called before the actual event finishes. So, on minimize, I had no problem, as I set the fake-child (FC) windows as children of the main window and they were all minimized together. However, on restore, things got ugly - by removing the parent/child relationship before the event finishes, all I achieved was preventing the FC windows from being restored. Out of desperation, I even did some experimentation with a timer, but, by this point, the "smell factor" was pretty much overwhelming.

So, inheritance it is. And how does it work?

I've created 2 classes: OrphanQMainWindow (which inherits from QMainWindow) and OrphanQDialog (which inherits from QDialog). These will be the two base classes for any app that needs to show the main window above (on the Z-order) its "child" windows.

OrphanQMainWindow is the simplest, but it dependes heavily on OrphanQDialog, so we'll show this class first.

We added the following (private) data members:

int zOrderIndex;
QPoint restorePosition;

static std::vector<OrphanQDialog*> orphanDialogs;
static unsigned int zOrderCounter;

orphanDialogs contains pointers to all the OrphanQDialog instances, until they're deleted. restorePosition is set when the dialog is hidden, so we can move it back to its position when it's restored. Without this measure, the dialog would be placed at the position it had when it was created. It's another design decision I don't understand, but let's not dwell on that.

zOrderCounter is used to keep track of the dialogs' Z-order. Each time a dialog is activated, zOrderCounter will be incremented, and its value will be assigned to the dialog's zOrderIndex. We handle this on the event handler for the Change event.

void OrphanQDialog::changeEvent(QEvent *event)
{
    QDialog::changeEvent(event);

    if (event->type() != QEvent::ActivationChange)
    {
        return;
    }

    if (isActiveWindow())
    {
        zOrderIndex = ++zOrderCounter;
    }
}

Next, we have three static member functions. These are called by the main window, when it minimizes, closes, and restores.

void OrphanQDialog::MinimizeAll()

{

    for (auto window : orphanDialogs)

    {

        window->restorePosition = window->pos();

        window->hide();

    }

}

void OrphanQDialog::CloseAll()

{

    for (auto window : orphanDialogs)

    {

        window->close();

    }

}

These two are self-explanatory.

void OrphanQDialog::RestoreAll()

{

    zOrderCounter = 0;

    std::sort(orphanDialogs.begin(), orphanDialogs.end(), [](OrphanQDialog* rhs,

        OrphanQDialog* lhs) -> bool

        { return rhs->zOrderIndex < lhs->zOrderIndex; });

    for (auto window : orphanDialogs)

    {

        window->move(window->restorePosition);

        window->show();

        window->zOrderIndex = ++zOrderCounter;

    }

}

Restoring is where everything comes together. First, we reset the Z-order counter. It's not really necessary, but it saves us from a highly-unlikely overflow, should our little app miraculously achieve an uptime measured in light years.

Then, we sort the vector according to the dialogs' ascending Z-order index. This makes sure we iterate through the vector to show the dialogs in the correct order, and also that we reset the dialogs' Z-order index without losing this order. As far as the user is concerned, everything is restored to its previous state, which was our goal, actually.

Finally, the ctor and dtor.

OrphanQDialog::OrphanQDialog(QWidget *parent) :

    QDialog(parent), zOrderIndex(++zOrderCounter)

{

    setAttribute(Qt::WA_DeleteOnClose);

    setWindowFlags(windowFlags() | Qt::WindowMinimizeButtonHint);

    orphanDialogs.push_back(this);

}

OrphanQDialog::~OrphanQDialog()

{

    // NOTE TO SELF: DON'T TRY TO USE RANGE-FOR WHEN YOU CALL

    // METHODS THAT NEED AN ITERATOR

    for(auto it = orphanDialogs.begin(); it != orphanDialogs.end(); ++it)

    {

        if (*it == this)

        {

            orphanDialogs.erase(it);

            break;

        }

    }

}

The dialog is created with the attribute Qt::WA_DeleteOnClose in order to avoid more logic to handle the case of dialogs that are still in the orphanDialogs vector but have already been closed, as we wouldn't want to show those. I've also added the expected frame buttons for a modeless dialog. And, yes, as you can see from the comment, I've tried the totally logic and even more totally wrong invocation of vector::erase() in the context of a range-for.

That's it for OrphanQDialog. Now, for OrphanQMainWindow, where we just implement the necessary events.

void OrphanQMainWindow::closeEvent(QCloseEvent *event)

{

    OrphanQDialog::CloseAll();

    event->accept();

}

void OrphanQMainWindow::changeEvent(QEvent *event)

{

    QMainWindow::changeEvent(event);

    if (event->type() != QEvent::WindowStateChange)

        return;

    Qt::WindowStates os = static_cast<QWindowStateChangeEvent*>(event)->oldState();

    Qt::WindowStates ns = windowState();

    // IF WE'RE MINIMIZING, MINIMIZE ALL THE OTHER WINDOWS

    if (ns & Qt::WindowMinimized)

    {

        OrphanQDialog::MinimizeAll();

    }

    //IF WE'RE RESTORING FROM A MINIMIZE, RESTORE ALL THE OTHER WINDOWS

    if (os & Qt::WindowMinimized)

    {

        OrphanQDialog::RestoreAll();

    }

}

Again, pretty much self-explanatory.

To get this working, you just add these classes to your Qt project, and inherit from them, instead of inheriting from QMainWindow and QDialog. And you may send my regards to Uncle Robert.

Two things to bear in mind:

• If you implement the changeEvent() event handler in your class that inherits from OrphanQDialog, don't forget to call OrphanQDialog::changeEvent(), or you'll lose the Z-order tracking. The same rule applies if you implement the changeEvent()/closeEvent() handlers on your main window class - e.g., if you implement closeEvent() and don't call OrphanQMainWindow::closeEvent(), the dialogs won't be closed when you close the main window.
• The dialogs aren't actually being minimized/restored, just hidden/shown. If your dialog classes need to do something when this happens, implement handlers for the hide/show events.

Points for future research:

• There might be a more Qt-ish way of doing this, perhaps using signals and slots.
• Take a look at how to actually minimize/restore the dialogs, instead of doing hide/show.

Ping again...

Still going through design, and plenty of PoCs (Proof-of-Concept).

And one - minor - rant...

An app creates a window. Call it a MainWindow, e.g., à-la-Qt. When the user works in that window, he may create other windows. If these windows are created as children of the main window, we have some tasks conveniently taken care of, such as minimizing/restoring those windows when the main window is minimized/restored.

However, in this arrangement, a child window can never be placed below (on the Z-order) its parent window; i.e., assuming no modal windows, when you click the parent window, it won't come to the top, it'll remain obscured by whatever child windows (children windows?) are on top of it. I realize this may be correct default behaviour. What I fail to understand is why is this dogma, why there's no property/setting to turn this off.

Why would I need to turn this off? My main window is what we might conceptually call a toolbox. On the course of executing a task, several tools may be needed, each tool corresponding to a window. So, the user may open a window (call it WinA), do some work and then need to go to the main window and open another window. It won't do to force him to move WinA out of the way to have uncluttered access to the main window.

I had this issue in C#, with WinForms. In the end, I had to create the windows as non-children and manage their behaviour myself. Now, with Qt's MainWindow/Dialog, I'm faced with the same problem. In C#, I created a separate class to track the main window and its "children". In C++, I wanted to try a different approach.

On my first attempt, I've created two static members in my fake-child (FC) window class:

• A counter. It's increased every time an FC window is created or activated. This counter corresponds to the FC window's Z-order index. This allows me to restore the FC windows in the correct Z-order.
• A container, to keep track of the FC windows.

With this setup, it was just a matter of implementing the ChangeEvent and CloseEvent on the FC window's class and on the MainWindow. Of course, I'd have to duplicate all this work the next time I needed it. So, I started looking for ways to avoid this duplication.

My first thought was a template class. After some experimentation, I've got a working solution for the Z-order that's mildly intrusive - requires implementation of a member function and a callback. I believe I can get rid of the member function, but not the callback - the event entry point is the parameter class, but the management occurs in the template class, which must be notified; hence, the callback. However, the template may not be the best solution, since I still need to implement the event handlers - this means we'd have more situations where the flow is from the parameter class to the template class. I'm still working on it, but I'm already considering other alternatives: 

• Inheritance. Instead of inheriting from QDialog, I could inherit from MyQDialog class, which would implement my solution. The same for MainWindow.
• An unorthodox idea that has popped up in my head - when the main window is minimized, set all the FC windows as its children, and when it's restored, set them as non-children again. I'd also have to set them as children when the main window is closed. If this works (i.e., if it has no side-effects), it might be the simplest solution, even though the "smell factor" is somewhat high.