Friday, 22 April 2016

Design - avoiding the obvious

Suppose you have a process that combs a database and generates monthly reports. The rule is simple, you run at the beginning of the month, and your universe is defined by the first and last days of the previous month.

See? Simple. You've got a begin date and an end date, and just grab everything that falls within that time period.

Since this date arrangement is a recurring requirement, I've decided it was time to abstract it into a script-lib, i.e., something other processes can call on the command line, and that can also be included as a library/module by other scripts.

And so I went about defining the interface, based on this use case - OK, we'll need a function that returns the begin date, and a function that returns the end date, and a rule for calculating the required interval. In my defense, I did kill the "and we could include formatting" thought as soon as it appeared.

Why all these functions? Well, SRP, and composability and all that jazz.

After a few attempts, I scrapped the idea. I wasn't happy with the interface, and I had no alternative design. I went on to do something else, and as I was looking at another problem, the solution hit me: The key is the first day of the current month. Everything else should flow from there.

This solution is in ruby (Ruby 1.8.7, no ActiveSupport), and it goes something like this:

def get_first_day_of_month(ref_date)
    day = ref_date.day()

    if day > 1
        ref_date -= day - 1
    end

    return ref_date
end


Now, with this, it's trivial to get the begin and end dates I actually need:

first_day = get_first_day_of_month(Date.today())


begin_date = first_day << 1 # subtract 1 month


end_date = first_day - 1 # subtract 1 day

Going back to my script-lib, I can now create an interface that can both return an array/hash with the two dates (usable as a module by other Ruby scripts), and output the dates to file/stdout, separated by a delimiter specified by the caller (usable by any script). Yes, it's a modest lib, but we all gotta start somewhere.

One thing I don't understand is the lack of modifier operators on date/time frameworks (see Java's Calendar class and Free Pascal's dateutils Recode* functions for two examples on how to do this right). It would've been a lot simpler to do something like this:

first_day = Date.today().day!(1)

Yes, I know, Ruby has open classes, I can add it myself. But just because I can, doesn't mean I should.

Thursday, 18 February 2016

Explicit template instantiation - Exhibit 1

As I promised last time, let's put our design to work on some code.

Back in 2013, I wrote a couple of template functions as a quick solution to a very specific problem - outputting non-ASCII characters on a Windows console in a program compiled with Mingw. Here's the header, and here's the source.

I've now applied the idea from the last post, and changed the code from full inclusion to explicit instantiation. There are other changes to be made, but those will wait.

If you go back in the header's history, you'll see that the previous version used full inclusion. As such, this was included in every translation unit (TU) that used this code:

#include "boost/locale.hpp"
#include "windows.h"
#include <cassert>
#include <locale>
#include <sstream>
#include <string>

Not necessarily outrageous. However, all of this was included because of a couple of functions. Not only that, but because the implementation was in the header, some helper functions had to be declared in the header, too, thus leaking implementation details.

Now, applying our new-fangled design, here's what we get.

Header (win_console_out.h)

Contains just the declaration for the functions.

template <typename CharT>
std::string ConvertOutput(CharT const* s);

template <typename CharT>
std::string ConvertOutput(std::basic_string<CharT> const& s);

template <typename CharT = DefaultCharT, typename T = void>
std::string ConvertOutput(T const& t);

Because the only #include we need is <string>, everything else has been removed from the header.

Source (win_console_out.cpp)

Just as before, contains the non-template helper functions. These are not part of the interface, and were removed from the header.

Implementation (win_console_out.ipp)

Most of the previous content of the header ended up here - all the #includes, the declarations for the helper functions, and the template functions definitions.

Explicit instantiation

Finally, each user of this code will supply its own explicit instantiation. In this case, you can see here what we're defining:

template std::string ConvertOutput(char const* s);
template std::string ConvertOutput(

    std::basic_string<char> const& s);
template std::string ConvertOutput(wchar_t const* s);
template std::string ConvertOutput(

    std::basic_string<wchar_t> const& s);

And that's it. I'll use my extensive (hah!) body of published code as a test for this design. If it works, I plan to keep using it, in order to reveal its ugly warts.

I've also created the mechanism for reverting to full inclusion via #defined variables, either globally or on a header-by-header basis. As I usually say, I believe users should have the right to choose, and I strive to keep to that principle.

Sunday, 14 February 2016

Explicit template instantiation as an organization tool

When you create a class template, the easiest way to organize your source code is to splat everything in a header file and be done with it. Users of your class just #include your header, and Bob's an annoying (in an endearing way) relative you get to see once a year.

Of course, the easiest way has a couple of drawbacks.
  1. Your header's code will get compiled for each translation unit (TU) that #includes it, even for similar types, i.e., if it has already been compiled before. During linking, all those equal copies (instantiations for the same types) will be discarded.
  2. Since your header contains the implementation, it's chock-full of... implementation details. Among other things, this means it probably #includes other headers. See #1.
This means your compilation time for class templates will put on some weight. Wouldn't it be nice if we could put it on a diet?

I've been taking a shot a this issue for the last few days, with the following goals:
  • The header should contain only declarations.
  • The implementation should go on a separate file.
  • The class template user should have as little work, and as many options available, as possible.
So, we'll begin with the header file. This file contains the class template declaration, and this is the file that will be #included by all the code that uses our class template.

Note: I'm leaving out the include guards, but they're necessary, same as in any other header.

listener.h
template <typename T>
class Listener
{
public:
    Listener();
    ~Listener();
private:
    T i;
};


We place the implementation in another file.

listener.ipp
#include "listener.h"
#include <iostream>

#include <string>

void print(std::string s)
{
    std::cout << s << '\n';
}

template <typename T>
Listener<T>::Listener()
{
    print("Listener:ctor");
}

template <typename T>
Listener<T>::~Listener()
{
    print("Listener:dtor");
}


Note we don't have to #include <iostream> or <string> in the header.

However, if we tried to build our program like this, it would fail, because the TU containing the header would not see the implementation. Which means we'll get missing symbols at link-time.

So, here we get to the first task for the class template user - creating an artificial TU to generate the code he needs.

listener_session.cpp
#include "listener.ipp"
#include "session.h"

template class Listener<Session>;


This explicit instantiation of the class template will generate all the template's members, which will allow the linker to find them and resolve the undefined references to those symbols.

Simple, heh?

Well, keeping the template declaration and definition in separate files is all fine and dandy, but if it were that simple, everyone would be doing it. There is a trade-off, and, as is usually the case, 80% of the time it’s probably not necessary; in fact, I suspect the old 80/20 rule may even be more skewed in this case.

Which begs the question – why am I spending time with this? Because that’s how I learn.

So, what’s the trade-off here? In order for this to work you must create explicit instantiations for all the types you will be using. So, instead of just #include-ing the header and defining your Listener<Whatever> variables, your project must have one or more TUs that #include both the header and the implementation and then contain explicit instantiations for the types you’ll be using, just as shown with listener_session.cpp, above.

And this means the projects that stand to benefit more from this practice are the ones where it requires the most work, namely, large projects.

The class template author can’t predict all the types that will parametrize the template, so he can’t supply the explicit instantiations (although sometimes he does, more about that in a minute). So, this means it’s up to the class template user to create the TUs with the required explicit instantiations. I happen to think the trade-off is worth it, but I’ve never been involved in a huge C++ project, so I could be wrong.

Has it been a minute, already? OK, let’s go to the case where the class template author organizes his code like this and supplies the explicit instantiations – when he wants to limit the types available to instantiate the template. By doing this, and withholding the source code, the class author guarantees that users of his template can’t instantiate it with any type other than those supplied.

So, we get the first two goals, but so far the third goal doesn't look good. There's not much we can do, within the language, to reduce the work required from our user. But we can give him more options.

Suppose our user doesn't actually care about all this. Maybe he's just developing a hello_template_world, or he loves coffee, and doesn't mind a 10 min. coffee-break every 30 mins.

In that case, we have several options.

Just include the .ipp file.
That's it, nothing more needs to be done.

Create a header the includes the .ipp file.
This allows the author some liberty with the naming of the .ipp file, if necessary. Other than that, it's just the same as including the .ipp file.

Quite simple, heh?

Yes, indeed. And quite wrong, too. We won't have much problem with the class template members, but that void print(std::string s) will send us straight into duplicate-symbol-land (BTW, this is actually the only reason why it exists).

So, how can we go about it? There's a simple solution (really), and it doesn't require that much extra work.

listener.h
template <typename T>
class Listener
{
public:
    Listener();
    ~Listener();
private:
    T i;
};

#if defined(LISTENER_FULL_HEADER) || defined(ALL_FULL_HEADER)
#define LISTENER_INLINE inline
#include "listener.ipp"
#else
#define LISTENER_INLINE

#endif

We give our users two options, they can either #define LISTENER_FULL_HEADER, which will only apply full inclusion to this class template; or they can #define ALL_FULL_HEADER, which will apply full inclusion to every class template.

And our implementation becomes this:

listener.ipp
#include "listener.h"
#include <iostream>

#include <string>

LISTENER_INLINE
void print(std::string s)
{
    std::cout << s << '\n';
}

template <typename T>
LISTENER_INLINE
Listener<T>::Listener()
{
    print("Listener:ctor");
}

template <typename T>

LISTENER_INLINE
Listener<T>::~Listener()
{
    print("Listener:dtor");
}


#undef LISTENER_INLINE

The #undef at the end is just for preprocessor hygiene.

How do we know this actually works as advertised? Here’s what we get when we build the program with each option. The code was built with VS 2015, and we used dumpbin /symbols on the resulting .obj files.

The results are edited, to fit on a single line.

Full inclusion gives us this:

ear.obj
1B0 SECT65 notype External Listener?3?5dtor?6?$AA@
1B4 SECT66 notype External Listener?3?5ctor?6?$AA@


main.obj
1C9 SECT6B notype External Listener?3?5dtor?6?$AA@
1CD SECT6C notype External Listener?3?5ctor?6?$AA@


The symbols are defined on the two object files, one of these will be dropped when linking.

Separate implementation gives us this:

ear.obj
020 UNDEF notype () External Listener<Session>::Listener<Session>(void))
021 UNDEF notype () External Listener<Session>::~Listener<Session>(void))


main.obj
075 UNDEF notype () External Listener<Session>::Listener<Session>(void))
076 UNDEF notype () External Listener<Session>::~Listener<Session>(void))


listener_instant.obj
199 SECT60 notype External Listener?3?5dtor?6?$AA@
19D SECT61 notype External Listener?3?5ctor?6?$AA@


We get the undefined symbols on ear.obj and main.obj, which will be resolved during linking.

So far, so good. Now, time to move this out of Proof-of-Concept-Land.

Monday, 8 February 2016

This behavior is by design

Visual customization (or personalization, or whatever else you want to call the ability to configure software to suit your specific needs) is a big thing for me. Especially for software that I use for long periods of time. And, for this kind of software, there's nothing more important than changing the color scheme. Sure, changing the font can be nice, changing the font size is definitely important, but changing the color scheme is, as far as I'm concerned, essential.

For me, there's nothing worse than staring at a glaring white background for hours. I've got nothing against all you 0xFFFFFF lovers out there, it's just not my thing.

These last few days, I've been comparing several visual customization alternatives, from the user's point of view. Why from the user's point of view? Because I consider that to be the most important point of view, we create software for users.

Also, when selecting the criteria for classification, there was one aspect that out-weighted the others by several orders of magnitude: Time. It's not just about changing the look of software, but doing it quickly.

As a final note, when I talk about visual customization, I mean the ability to change just about every visual aspect of the interface. There is a name for software that has a "customization" option that only allows you to change the color of the title bar or the toolbar, but I won't say it here, there may be children reading this (I hear it's great for insomnia).

So, here's the final result, from best to worst:

1. The software provides official alternatives to the default look

"Official" here means "created or curated by the software developer", and easily accessible/installable via some sort of manager that takes care of download/installation. To qualify, these alternatives must be diverse (not just variations on glaring white) and must actually work (e.g., no blue text on a black background - yes, Linux shell, I'm looking at you).

Oh, and "curated" means someone actually looked at the whole thing and confirmed that we're not getting the blue-on-black stroke of genius. Being free of bugs/exploits/nasty surprises is a must, but it's not enough.

For software that makes this grade, I don't really care that much about how difficult it may be to change individual aspects of the provided looks/themes/whatever, because we have a coherent whole that works.

Out of the software I use, the winners are Visual Studio and Qt Creator. Congratulations to these teams, top quality work. Android Studio follows right behind, and the only reason it's not up there with those two is because does have some hard-to-read combinations, where the text and background colors are very similar.

2. The software is easy to configure

Since we're not guaranteed to have a coherent alternative here, it must be easy to either change the whole look (e.g., via theming) or individual aspects of it.

So, here we may have software that has a "wealth of community-provided looks/themes/whatever", where trying out a theme is trivial, but changing each individual aspect is not - e.g., Chrome/Firefox extensions.

And we also have software that may or may not have all that wealth, but has a trivial way of either changing the whole look or individual aspects of it - e.g., Notepad++ or gedit.

Changing individual aspects may not give the user a complete coherent workspace, but it will provide a good starting point and, since it's easy to configure, it will allow the user to quickly solve problems as they appear. Again, a very time-efficient solution. Not as good as #1, but a positive experience, all things considered.

3. The software is not easy to configure

Here, it's irrelevant if we're talking about the whole look or individual aspects of it.

This is the software that expects the user to manually download some sort of file (usually, an archive file), copy it to some directory and expand it; or to copy some existing files from a system-protected directory to a user directory and then edit those, looking for some more-or-less cryptic configuration key and changing some usually-even-more-or-less cryptic value; and then, maybe, having to restart the software. Bonus (negative) points if it's a system-wide configuration, where "restart" actually means "reboot".

Most Linux desktops I've tried fall in this category. In order to change a simple visual aspect, if you can't find a recipe for changing the exact item you want, you're in for a good reading of Gtk, or Qt, or some-other-widget-toolkit docs (assuming what you want is properly documented), followed by some file copying, key-searching, and value tweaking. And, since what you'll get will usually be a good starting point, you'll have the enormous pleasure of repeating the process as further problems appear.

Oh, and if you do find the recipe you're looking for, check its date, and make sure the toolkit/software version match yours.

Here, I usually do just enough work to splash a dark background on the software I use the most and ignore everything else. I definitely don't want to waste more time than absolutely necessary.

4. The abomination

This is a special category. You may have already got that impression from the heading, but it's also special in that it has only one entry: Windows Aero.

Let's make this clear - I often look at design options and think "This is incredibly stupid, but maybe there is some logic that I'm missing here". As the man sang, "I'm not the sharpest tool in the shed", so there's definitely room for error on my part. However, when we get to Windows Aero, I can't get past the comma in that sentence. I've considered it several times, and I can see no logic here.

Let's look at the symptoms:
  • Countless posts asking what should be a very simple question, "How do I change the background color for Windows Explorer?", and getting one of two answers: Either "Switch from Aero to Basic/Classic/Anything-Else-That's-Not-Aero" or "Why would you want to do that, there's plenty of people that use white and love it". Both these answers actually mean "You can't".

  • Of course, that's not entirely true. You can. You just have to reverse-engineer the binary format that stores the color configuration (basically it's a DLL containing resources). Or pay for a utility created by someone who went through that effort; then, you'll still have to create your "visual style", and you'll still have to patch Windows to allow you to use your non-signed "visual style".

  • Yes, you read that correctly. Binary format? Check! DLL? Check! Patch Windows? Check! Options such as the background color for applications are stored (although "buried", "concealed", or even "encrypted" would be more suited here) in an undocumented binary format, in a DLL, stored in a protected system folder.

This is the only example I've found where your best option is to actually pay for a third-party application to do something as simple as changing a background color. BTW, this is also why you find successful commercial alternatives to Windows Explorer. It's not just the extra functionality these alternatives have (and they do add value to their offerings), it's also this brain-dead design by the Aero team, that means that something as simple as "the ability to change the background color" is a value point.

Here, I don't even waste my time. I'm just grateful that whoever designed Windows Aero didn't get to unleash its remarkable genius anywhere near Visual Studio.

Wednesday, 6 January 2016

Logging Abstraction Layer - Take 2

As I did on the previous post, I'll begin by saying I won't go into another "My, how long it has been" post. I do, however, hope I'm not too late to express my Best Wishes for 2016.

Now, then...

I'm having another iteration on my idea for an abstraction layer for logging libraries, which is basically fancy-speak for “a collection of (hopefully, organized) macros” that will allow me to replace logging libraries with minimal effort.

I've been going back and forth between Boost Log and Poco Logger. I like both, even though I tend to prefer Poco Logger, because I haven't yet found a satisfactory solution for the issue with Boost Log's preference for truncating an existing log file when the application is restarted.

I've hit some maintenance problems with my “collection of macros”, so I decided to give Boost Preprocessor another go. Back when I began working on this idea, I looked into Boost PP, but got caught in a blind spot, probably a faulty reaction to all the metaprogramming examples. I had the feeling I could use it, but I wasn't able to make it work.

So, I rolled my own and have since been banging my head against several walls, thus gaining the required experience to make a number of questions I didn't even know were there. Which gave me a different perspective when I approached Boost Preprocessor again, and things went much better this time.

I quickly set up a PoC, and then I hit the issue I'm looking at now – How to format the log record. This is only somewhat important for readability, but it's crucial for automation. Without a structured format, we'll have a hard time creating tools to operate on the logs.

My initial design, which I haven't changed, treats the logging statement as a sequence of values to output. E.g.,

SOMEPREFIX_DEBUG(object_id, discount_id, “Getting object price”);

But a structured format requires other artifacts – at a minimum, we need delimiters. And an obvious requirement for some sort of intelligent automatism is that the nature of the values must be identified, i.e., we need to know if an element logged as an integer is, say, a total or an ID; so, we need to decorate values with names.

Boost Log gives us attributes, values which are added to the log record without the need to output them explicitly on the logging statements. Furthermore, attributes can be used in filtering and formatting, and can be scoped. I suppose there may be even performance benefits in the use of attributes, but that's something I'd have to measure.

At first, I thought this could be the way to go. Define attributes, create formatters or override the output operator, and control the way the record is written to file.

However…

Poco Logger does not have this concept. And while I'm not considering any other logging library at the moment, that may change in the future. And, as I said above, my main goal is to “replace logging libraries with minimal effort”. So, if this is my main goal, it is important enough to warrant an under-utilization of the library.

Naturally, if I don't use the library's capabilities to output this data, I'll need to output it on the logging statements. This means that formatting will also have to happen there. And since I'm already using the preprocessor as an abstraction mechanism, I'll just… use it some more.

So, I already have something like this:

SOMEPREFIX_DEBUG(object_id, discount_id, “Getting object price”);

What I need is something along these lines (simplified):

SOMEPREFIX_DEBUG
(
    SOMEPREFIX_FORMAT(“ObjectID”, object_id),
    SOMEPREFIX_FORMAT(“DiscountID”, discount_id), 
    “Getting object price”
);

Which could, then, output something like this (line breaks for readability):

2016-01-01 00:00:00 <DEBUG> 
    { "ObjectID": "42", “DiscountID”: “24” } Getting object price

Then, one day, if we decided life had become too easy, we could switch formats to something like this, without changing the logging statements:

2016-01-01 00:00:00 [DEBUG]
    <LogRecord><ObjectID>42</ObjectID>
    <DiscountID>24</DiscountID></LogRecord>
    Getting object price

So, the next step will be identifying the required formatting elements, and how to incorporate them in the logging statements. The goal here is to keep the logging statements as simple as possible.

On this iteration, I will leave out any sort of container/compound data type. Not only will these make the design a lot more complex, but I'm prepared to do without them – in my experience, I have found very few scenarios requiring the logging of these types, and it has always been possible to find a workaround somewhere between acceptable and undesirable.

Tuesday, 13 October 2015

Visual Studio 2015, ICU, and error LNK2005

I'll begin by saying that I'm just going to ignore the fact that I haven't written anything in nearly nine months.

So...

While building ICU 56.1 with VS 2015, I was greeted with thousands of errors like this (also described here by someone who came across the same problem):

error LNK2005: "public: static bool const
std::numeric_limits<unsigned short>::is_signed"
(?is_signed@?$numeric_limits@...@std@@2_NB) already defined in
ParagraphLayout.obj

This is defined in <limits>, in a statement like this:

_STCONS(bool, is_signed, false);

Looking at the pre-processor output, we can see its actual definition:

static constexpr bool is_signed = (bool)(false);

If I understood the Standard correctly, this should be OK, and there should be no duplicate symbols during linking. So, I was still missing a logical cause for this.

The usual internet search for «ICU LNK2005» didn't bring anything useful, except for the link above.

Then, as I concentrated my search on LNK2005, I came across this post. The same mysterious behaviour, but now there was a plausible explanation, in a comment by MS's Stephan T. Lavavej, in a quoted post from an MSDN blog:

We recommend against using /Za, which is best thought of as "enable extra conformance and extra compiler bugs", because it activates rarely-used and rarely-tested codepaths. I stopped testing the STL with /Za years ago, when it broke perfectly conformant code like vector<unique_ptr<T>>.  
That compiler bug was later fixed, but I haven't found the time to go re-enable that /Za test coverage. Implementing missing features and fixing bugs affecting all users has been higher priority than supporting this discouraged and rarely-used compiler option.

So, after removing /Za from all projects in ICU's allinone VS Solution (Project Properties -> Configuration Properties -> C/C++ -> Language -> Disable Language Exceptions -> No), I was able to build it with no errors, on all configurations (x86/x64, debug/release).

Apparently, it's one of those rare cases where the error is actually in the compiler, not in the code.

Saturday, 31 January 2015

CA Certificates - The tale of the invisible certificate

I've been through a memory upgrade on my 5-year old PC. My goal is to set up a few VMs running simultaneously, because I need to widen my scope for experimentation. I found out my BIOS has an incompatibility with the memory DIMMs currently available, but fortunately a friend lent me 8GB, so I can start working now, while I try to sort out this mess.

As I set up each VM, I'm importing my bookmarks, so that I have my net environment available "everywhere". And I've come across a curious situation, regarding certificates.

One of the URLs I have on my bookmarks is https://www.ddo.com/forums. The first time I accessed it on Firefox, I got an error message:
Peer's certificate has an invalid signature. (Error code: sec_error_bad_signature)

Using openssl s_client, I checked that ddo.com sends only its own certificate, not the chain, so I looked up the chain in IE, and checked the intermediate CA on Firefox's certificate store. It was there, but it was a different certificate - different signature, different validity (both valid, because the validities on both certificates overlapped), different issuer, only the subject was the same.

I exported that CA certificate from IE, and ran openssl verify using each CA certificate; using the one from Firefox certificate store, I got an error; using the site's CA certificate, the validation succeeded.

So, I imported the site's CA certificate to Firefox, accessed the site, and all was well again.

Then, I checked Firefox's certificate store. And I only found the exact same certificate that was there already, and which wasn't previously validating ddo.com's certificate. Except that now it was.

And much scratching of head ensued.

Until yesterday, when discussing this at lunch with a friend, he told me the obvious: "Well, if you imported it, and the site's certificate is now correctly validated, then it must be there, even if you can't see it". And that gave me a memory jolt, to an issue I had a little more a year ago, with Sun One's web server certificate store, where we had two certificates for the same CA, but only one was visible on the web console. In order to correctly see both, I had to use certutil on the command line.

And in this case, the solutions was the same:

certutil -L -d sql:path to Firefox profile directory -n certificate subject

Which promptly listed the two certificates.

And another Mystery of the Universe was solved during a meal.

I don't understand why the GUI shows just one certificate. I'm not going to say it's stupid because it may be a reasonable decision, based on knowledge I don't have. But to completely hide the fact that a CA has two simultaneously valid certificates on the store is terribly misleading, it's definitely not what I'd call a good solution.

In the end, it was command line to the rescue... as usual.

Saturday, 10 January 2015

Visual Studio - Getting to your debugging symbols

I've recently been reformulating my lib environment. Basically, it consists of:
  • A loading zone, where I install the library sources, and run the build process. It has a folder for each lib, with each version in its own sub-folder.
  • A library root folder, with sub-roots for mingw and MSVC, where I store the files resulting from the builds. Again, each lib folder has a sub-folder for each installed version.

There's a bit more to it, but it's not important for today's post.

This reformulation has gone through several iterations, and it's still a work in progress. This time, I had the following goals:
  • G1 Getting some further progress on automating the process of building libraries from source, both release and debug versions. Actually, it was this requirement for debug versions of all the libs that led to my patch to OpenSSL with a debug configuration for mingw32.
  • G2 Correcting some bad decisions regarding the lib folders' names, especially when it comes to version numbers. The goal is to use the same number format each lib uses for its own files, where applicable.
  • G3 Clearing the landing zone after building the libs. Now, when I finish building, I run make (actually, mingw32-make or nmake) clean.

Point G3 led me on another learning experience.

The most common debug option when using MSVC seems to be /Zi, which stores the debug symbols in PDB files. As such, I haven't looked into the other options, which store debugging information in the object files themselves; I won't discuss them in this post, but if I had to hazard a guess, I'd say the behaviour is the same as the one described below.

During a debug build, the compiler/linker stores the absolute path to the PDB file on the executable files (EXE or DLL). When you fire up the debugger, as it loads these executable files, it looks up the PDB using this path, to load the symbols. If it can't find the PDB, it then goes through other steps.

On the open-source projects I've seen, I've come across two default behaviours - either the PDB files are not copied to the library folder at all (e.g., Boost); or they're copied to the folder containing the static libraries/import libraries, but not the DLLs themselves (e.g., ICU). I call these default behaviours, because there may be options for gettting a different behaviour; I've looked for these, but found none.

This, combined with point G3 above, is not-so-good news, because make clean deletes the PDB files. So, when we fire up the debugger, it cannot find the symbols for these DLLs. It never happened before because my procedure has always been:
  • build release.
  • clean up.
  • build debug.
  • don't clean up.

So, the PDBs were always available at the landing zone, i.e., the path stored on the executables. And even though some libs copied them to the lib folders, these weren't actually used when the DLLs were loaded by the debugger.

There are several options for dealing with this, including these three:
  • O1 Setting up a local symbol server, which, as I understand it, is more of a local cache to store symbol files.
  • O2 Adding each individual folder where PDB files are installed to the _NT_SYMBOL_PATH env variable, which MS debuggers use to locate symbol files.
  • O3 Manually copying the PDB files to the DLLs folders.

I believe option O1 is the best, but I'll leave it for another iteration. For now, I'll go with option O3. Like I said, this is a work in progress, and I don't feel a particular pressure to go for the optimal solution (which I'll have to test before I adopt it), I prefer getting to a working solution faster, in order to keep my self-imposed deadline (which will end this weekend).

Of course, once you have all this worked out, you still need your debugger to load the correct DLLs. You may have other versions of those DLLs on your path; with popular libraries, like OpenSSL, this is more common than you may think.

On Qt Creator, assuring that you load the correct DLLs is very simple. On Projects mode (Ctrl + 5), you switch to the Run configuration and edit the PATH on the Run Environment. Since I don't have these libraries on the PATH, I always need to do this; if you usually have your debug libraries on the PATH, you don't need to edit anything.

Visual Studio is a different sort of creature. A larger sort of creature, where everything usually takes a bit more work to find.

I knew it was on Project Properties (Alt + F7). At first, I thought it was on Configuration Properties -> VC++ Directories -> Executable Directories. When I hit help, it took me here, where we can read: "Directories in which to search for executable files. Corresponds to the PATH environment variable". Fine, that's just what we need. Somewhat later, and after a moderate amount of gnashing of the dental (not mental, mind you) persuasion, I noticed that the description on the project Property Pages was a wee-bit more complete, namely "Directories in which to search for executable files while building a VC++ project". Ah... right... building the project... as in, "not running the executable".

Then, I turned to the next obvious choice, Configuration Properties -> Debugging -> Environment. This time, I read the description before going for the help button. It is quite helpful, it says "Specifies the environment for the debugee, or variables to merge with existing environment". After reading this, I knew exactly what to do. Which was hitting the help button, and hoping the help page for this option was more useful.

Fortunately, it was. We control the PATH here, using something like this: PATH=E:\Dev\lib\msvc\openssl\openssl1_0_1j\debug\bin;E:\Dev\lib\msvc\icu\icu54_1\debug\bin;%PATH%.

And, finally, after some gnashing of the mental persuasion, I can say that Visual Studio's debugger and me are finally getting along just fine.

As they say, until next time... when I turn this into reusable project configurations, instead of having to specify these manually on every project.

Thursday, 8 January 2015

OpenSSL - Debug build configuration for mingw32

I've just submitted a patch to OpenSSL, to include a debug build configuration for mingw32, as it currently lacks one.

Not only have I done the obvious - removed optimizations and add the -g compiler option, but I've also used debug-Cygwin as a starting point to include a set of debug #defines for OpenSSL. You can find the patch here.

At first, I sent it to the openssl-dev mailing list. Later, I find out there was already an issue open for this on OpenSSL's RT.

Anyway, the patch makes the changes below.

config

This is the script called from the command line for configuration, which will generate the makefile.

The patch adds MSYS* to the script's platform guessing mechanism, allowing us to build on MSYS2. It wasn't really necessary for the debug configuration, but I like to have the ability to build on MSYS2.

Configure

Configure is a perl script that is called by config.

The patch adds the mingw32 debug configuration (debug-mingw), and changes the part of the script that defines the executable extension, to look for /mingw/ instead of /^mingw/, which then makes it work correctly with both mingw configurations.

As a side note, I think it would have been better to name the debug configurations as <platform-debug>, rather than <debug-platform>, but I don't know the reasons behind this, so I might be wrong.

Makefile.org

This file is a base for the final makefile.

The patch changes platform checks from mingw and mingw* to .*mingw and *mingw*. Curious, now that I'm writing this, it's the first time I've noticed the incoherence. I'll probably go back to the testing board and change that to *ming* on both.

Makefile.shared

This file is used when we're building shared libraries (DLLs on mingw).

This is where I was more bold with the changes. The original file was always passing -Wl,-s to the linker, meaning it would clear the symbols from the object files. However, on a debug build we want to keep the symbols, that's the main reason for wanting a debug build in the first place.

So, the patch not only adds a check for .*mingw instead of mingw, it also changes the linker arguments for the debug configuration, excluding the -s.

And that's it. Not that much work, but a lot of experimenting. And, since shell script and make are not the most user-friendly debugging experiences, it has required a lot of creative experimenting.

Hope someone finds it useful.

Saturday, 27 December 2014

'Tis the Season...

... to be jolly, and grateful, and hopeful.
 

On the personal front...

...this year has been very positive, the only clouds on the horizon being some health issues. If I had to choose the most important points of 2014, I'd go with these:
 
  • The level of complicity and understanding between me and my wife has increased tremendously. I believe we achieved this by undergoing a similar change, which takes us right to the next point...
  • We both began going at life with a more relaxed attitude. It's a daily exercise to keep from slipping, but it's quite worth it. Obviously, it occasionally slips, but not only have we gotten better at identifying these slips, both on ourselves and on each other, we are also quicker to defuse these situations, often by sharing a good laugh at ourselves.
The kids are working their way through college, on what are the first steps of their own journey, their own adventure. It's time they take full control of the pen and start writing their story; that's their part. Our part is hoping everything they've taken in through the years will serve them well in getting their bearings as they set out. More than hopeful, we're confident it will.
 
I'm a somewhat spiritual guy, although I don't often stop to think about it. However, looking at 2014, I feel blessed; I already knew I'd found someone more understanding of my failings than I ever deserved, but this year took it to a new height. If I have so much to write about in the following point is because I was fortunate enough to find someone as understanding as my wife.
 
If you don't care much for mildly technical stuff, you can jump straight to the end of the post.
 

On the professional front...

...this has been a year where I continued a trend that began in mid-2013, namely, generalization.
 
When it all began in 2011, I had picked up C#. Then, because I felt I wasn't learning anything other than the language, I've added C++. Actually, my plan was adding C++, but I ended up switching to C++, and C# was left behind. The learning experience was a lot more intense, not just about the language, but also about the whole system - processor, OS, environment, external libs, debugging, assembly, and a whole lot of etc.

Then, I've accidentally set out on a task for which I found little help on the web. So, for much of what I was doing, I was on my own. While I did manage to get a working result (which is still a top-hitter on google.com and bing.com), I wasn't totally happy with it, and I suspected I'd have been even less happy if my knowledge about what I was doing wasn't so lacking. So, I've stepped back and went back to basics. And I've quickly learned that, indeed, the potential for "improvement" (i.e., correction) in what I'd done was much bigger than I had anticipated.
 
Then, as I began taking on more technical tasks at work, the generalization began - certificates, network, DNS, managing Linux, setting up environments according to specific constraints, managing web servers, managing Weblogic server, picking up new languages (e.g., Ruby), revisiting familiar languages (e.g., Java).
 
At the same time, I've taken the learning experience provided by C++ deeper - assembly debugging, system traces, building gcc from source and installing it without touching the system gcc, doing the same with perl. And I've also began getting my feet wet with Javascript (bootstrap and query.js) and Android development.
 
And before I noticed, I had not only changed my course, but I was quite satisfied with that change. This is where I see myself going, becoming a generalist. I love solving problems, and you need a widespread knowledge to do it; you don't always find the cause of a problem at the same level, and I don't like getting stuck for lack of knowledge. I also don't like getting stuck for lack of system access, but that's the way things are when you work in a large-ish corporation.
 
So, here are my New Years resolutions, a.k.a., goals for 2015:
  • Redesign my libssh2 + asio components, incorporating what I've learned in the meantime. And hoping that two years from now I may look at what I've done, say "What I was thinking??!!", and repeat this goal again, as I keep learning more.
  • Pick up a functional language, probably Haskell or Erlang. It's about time I get my feet wet on functional programming. I love what I've learned about generic programming in C++ (actually, what I'm still learning), but it's time to add something more to the mix.
  • Continue my exposure to Javascript and Android.
  • Deepen my knowledge of systems/network administration.
  • Increase my knowledge of low-level tracing/debugging. It's time to begin some serious experiments with loading up core dumps and getting my bearings, and getting more mileage out of stuff like Windows Performance Analyzer.
Too ambitious, you say? Yes, I know. I won't manage to do all of this? Probably not. Which is a good thing, otherwise 2016 would be a very boring year.
 

At the end of the day...

 
 
...our family wishes you and your loved ones a Merry Christmas and a Happy 2015, filled with love, joy, and peace.
 
 

Sunday, 7 December 2014

A simple class to read files

Yes, I'm olde enough to know "simple" is a misnomer.
 
I have a new addition to my bluesy github repo.
 
Part of what I do at work involves reading through user-un-friendly log files. How user-un-friendly? Just to give a small example, the cases where a "log record" is on a single line are the exception (I wish it was just the Exceptions), and there's usually no easy way to correlate the beginning/end of operations with a single grep. This means I spend a lot of time creating scripts/small programs to process those log files.
 
I'm not an enthusiast of shell scripting. Oh, I can persuade it to do some complex stuff, yes, but it always seems to obey me reluctantly, and I'm left with the nagging feeling that, somehow, it's laughing behind my back. Not to mention that the resulting scripts are very... spaghetti-like. I find it hard to think modularly when writing shell scripts. I had a similar issue with Perl. It's still there with Ruby, but being modular with Ruby is a lot easier for me.
 
However, I don't have this difficulty with C++. Or Java, or C#, or Delphi/Lazarus, for that matter. It must be some sort of mental block, because while I couldn't create a modular design in shell scripting to save my life, that comes as second nature when I'm working in C++.
 
So, when I have to create some kind of custom tool to process these files, most of the time I turn to C++. Thanks to C++1y/Boost, I don't take much longer to write it than I would with any other tool/language, and I definitely appreciate that when I need to change something a few weeks/months later, I can find my bearings much more quickly.
 
And, after creating a few very similar programs, all driven by a file-reading loop, I've figured I had enough use-cases to create...
 

FileLineReader

A class... actually, a class template, that reads lines from a file. Surprising, heh?
 
template <typename LineMatcher = SimpleLineMatcher,
    typename LineCounter = SimpleLineCounter<unsigned long>>
class FileLineReader : private LineMatcher, private LineCounter
 
The concept is simple - read a line, which becomes the current line, and give the caller a way to get it (or copy it). Then, we add some nuggets of convenience:
  • Keep a counter of read lines. Implemented by LineCounter.
  • Supply matching operations, that not only perform matching on the current line, but also allow skipping lines based on matching/non-matching. The matching is implemented by LineMatcher; which is then used by FileLineReader to implement the skipping.

Why inheritance, instead of composition? Because there will be cases where LineMatcher and LineCounter have no state, and a data member is a bit of waste (yes, a tiny little bit of a waste). Can this be abused? Absolutely, but you know - protect against Murphy, not Machavelli.
 

Skipping lines 

The first line skipping functions I introduced were SkipMatchingLines(), which skips lines while there's a match, and SkipLinesUntilMatch(), which skips lines until it finds a match.
 
These functions share a similar trait - their stop condition is met after reading the line that triggers the stopping condition. Suppose we have this file:

[2014-01-01 00:00:00.000] match-1 This is line 0
[2014-01-01 00:00:00.100] match-1 This is line 1
[2014-01-01 00:00:00.200] match-1 This is line 2
[2014-01-01 00:00:00.300] match-2 This is line 3
[2014-01-01 00:00:00.400] match-2 This is line 4
 
Something like this

FileLineReader<> flr{kFileName};
flr.SkipMatchingLines("match-1");
// Process lines
 
can only stop when the line "... line 3" is read, because there's no way we can perform a match against a line that hasn't been read yet. This means that when we reach "// Process lines", the current line will be the first line to process, so we should "process-then-read" (do-while), rather than the more intuitive "read-then-process" (while).

I've entertained the notion of using tellg()/seekg() to rewind the file (IOW, un-read the line), but I didn't even get started, after reading about how it behaves in text mode. So, I'll stick with "process-then-read", for now.

Another common scenario I encounter is skipping a certain number of lines, usually a header. So, I've added this:
   
void SkipNumberLines(unsigned int number_lines);

Because we're skipping lines independently of any matching, it didn't make sense to implement this in LineMatcher; so, I've implemented it straight in FileLineReader. I'm not completely happy with this solution, but I figured it's better than no solution.
 

Skipping dilemmas

I also thought it would make sense to keep semantic coherence between all the line-skipping functions. So, SkipNumberLines() should stop on the first line to process, not on the last line skipped (even though it could, because we're skipping a known number of lines), just like the other functions.
 
No biggie, we just perform an extra read - so, for SkipNumberLines(3), we actually read 4 lines. Lovely, everything's coherent, little ponies are happy, and fairies are spreading pixie dust and sneezes throughout the realm.
 
The thing is... nobody expects the Spanish Inquisition... their chief weapon is surprise and SkipNumberLines(0)... their two chief weapons are surprise, an almost fanatical devotion to the Pope, and SkipNumberLines(0)... OK, among their weaponry we can find elements as diverse as... blah, blah, blah, and SkipNumberLines(0).

Yep, SkipNumberLines(0).

Let's say you have a function that calculates how many lines to skip, based on, e.g., an input argument - it could be the type of file being processed, or the phase of the moon; then, you could use it like this:

flr.SkipNumberLines(
    CalculateNumberLinesToSkip(some_relevant_argument));
 
You're aware CalculateNumberLinesToSkip() could return 0 (you could, say, be processing a file without a header). And I'd venture a guess that you would probably expect that SkipNumberLines(0) would skip, you know, a number of lines kinda equal to... zero. As in, more than -1, but definitely less than 1.

Which left me with two alternatives:
  1. Treat 0 as a special case. Which would force upon the caller the "specialness" of this case.
  2. Indulge in a fanatical devotion to coherence - read one line on SkipNumberLines(0), and consequences be damned.
  3. Forget coherence, and accept that even though they're all line-skipping functions, they can have different semantics/post-conditions.

OK, enough Spanish Inquisition jokes.
 
I've settled for alternative 3. I'm not entirely sure it's the best option, and as I get more use out of this class, I expect to find new scenarios and patterns, which will provide me with more data to revisit this design later. But, for now, it seems to be the safest choice.

And there you have it. Quite modest, yes, but it has been saving me some boilerplate code in these last few weeks.

I don't know yet how it will evolve. I can feel an itch with regards to a text mode rewind, but I'll have to dive into streams and buffers much more deeply than I feel like, at the moment.

And right now I have other challenges awaiting... impatiently, I might add. A crash course in some obscure aspects of Weblogic administration to better diagnose socket problems without resorting to strace.

 

Sunday, 14 September 2014

Performance puzzle(d)

So, back to programming.
 
Every now and then, I pick up a programming puzzle. The goal is not only to solve the puzzle, but also the learn more about the performance of my solutions.
 
A few days ago, I've decided to take a shot at making a Cracker Barrel solver. Simple stuff, just brute-force your way through solving the puzzle. First, I've started with a standard 15-hole board. Then, I've moved on to a configurable board (but never smaller than 15 holes). Then, I've implemented getting the board setup (including the list of all possible moves) just from the number of holes. Then, I've decided that this list would be kept in an std::array, which meant getting this information at compile-time, which gave an excuse for a little bit of basic recursive template meta-programming lite.

Yes, I've forced feature-creep on myself. Ah, and no design optimization - e.g, I didn't use STL's bitset<N> (or even C bit-twiddling) for the board, and I've created a good-ole struct for the moves, with three integers.

I've built it with GCC (mingw32/Qt Creator) and MSVC 2013. For a board with 15 holes, both were immediate. As I've moved to 21 holes, the GCC exe took a few seconds, but MSVC took almost... 1 minute?! With 28 holes, Qt Creator takes forever, and so does MSVC.

The fact that this takes forever doesn't surprise me. As I said, I've made no optimizations. However, the difference between GCC and MSVC puzzled me. So, I turned to Windows Performance Analyzer (WPA) to figure out what was going on.

Now, I had a good guess about what was going on - I was sure that, compiler optimizations notwithstanding, there was probably a lot of copying going on. There certainly is a lot of vectors getting constructed, and I was expecting to find my main culprit along those lines.

So, I fired up WPA, loaded the trace file, selected the CPU Usage (Sampled) graph, changed it to Display table only, opened the View Editor and set the Stack (call stack) to Visible.

And this was flagged as the most time-consuming operation of all:

std::_Equal_range<GameMove const *,GameMove,int,
    bool (__cdecl*)(GameMove const &,GameMove const &)

Sometimes, life proves more interesting than anticipated. However, this time it was just me not paying attention.

When I say I've made no optimizations, I didn't even go for the most important optimization of all - an intelligent algorithm. This means that when I calculate all the valid moves after each move, I go through the entire board, including spots for which no move is possible. And I'm using sorted vectors/arrays and equal_range() to perform this search.

So, while it the result was surprising, it shouldn't really have been.

Next step - find a more intelligent way to get the list of valid moves.
 

Saturday, 30 August 2014

Extremely destructive

Off-topic, today. Quite so, actually. If you came here to read about programming, you may want to give this one a miss.
 
There's this light comedy French film called "Le Placard" ("The Closet" in English, "Sai do Armário" in Portuguese). Quite enjoyable, BTW, recommended.
 
The IMDB summary says it all:
A man spreads the rumor of his fake homosexuality with the aid of his neighbor, to prevent his imminent firing at his work.
The man's elderly neighbour is actually gay, and at some point mentions the irony of him being fired for being gay back in the day, when he was young, and how these days what happens is the exact opposite, because of anti-discrimination laws/fear of lawsuits/whatever. Yes, I know it's an over-simplification, but it's a light comedy, not a documentary.
 
However, it's no secret that, if we go back a few decades, being gay could land you in trouble at the workplace (and land you out of the workplace). It was a totally unfair practice, where your professional performance would be ignored and you'd be judged exclusively on (selected parts of) your personal profile. And you have any doubt on how unfair and tragic this was, just read Alan Turing's bio, and think on how many "anonymous" people suffered similar fates.
 
Fast-forward to the 21st century, and things are looking a lot better... or so I thought. Until I witnessed the whole Brendan Eich's debacle.
 
Speaking solely for myself, the only thing it has achieved was to make me more cautious in providing my support to a cause I've always considered worthwhile (and still do) - equal rights for everyone. That's the way I've always voted, and that's the way I've always seen myself voting. But now that I've witnessed how the previously-persecuted can do the exact same thing to others with such abandon and glee, I'm left wondering - if I'm ever called to vote on any subject-related matter, what will I do? As the guy sang, "I don't know the answer".

You see, the thing is... I don't like mobs, and I'm not particularly amenable to the argument "We're a mob, but it's for a good cause, a just cause". There is no "good cause" in mob mentality, only rabid persecution.
 
However, this incident was months ago, why write about it now? Well, because now we have "the mobs" up in arms again, this time in the "gaming industry".
 
I'm an on/off gamer. I'm a Steam customer, and a very good GOG customer (if there's one thing I've learned is to vote with my wallet and reward those that actually care about their customers). These days, I sometimes go for months without playing a game. Before I got back into programming, I could spend all my free time playing games. Now, gaming is relegated to 3rd place, behind programming and guitar-playing (I'm excluding Real Life, which means family, friends and work, obviously).
 
I've never had any problem in calling myself a gamer, even as it stopped being part of what I did. And then, slowly but surely, the word "gamer" began to take on very specific meanings, all of them negative. It didn't bothered me much. It's like reading "All Portuguese are lazy bastards, living off everyone else's money", this says more about the speaker than about the Portuguese people.
 
Anyway, now it all came to a climax with this last brouhaha. I'm sure you can find the... "relevant information" (for lack of more appropriate words) and links. I'll just link to the best summation I've seen of this whole deal:
So many interesting conversations and things to challenge. But you can't have a discussion during a riot. It's a shame.
Indeed. What you have here is a) a few people who should be detained for online harassment/threats; b) some people on both sides who have nothing to say except indulge in some feces-throwing (tribalism at its best); c) some people (fortunately, more numerous), again on both sides, who are raising up important issues that should be discussed (but, as Shamus put it, not "during a riot"); and d), (going out on a limb here) I'm willing to bet you have an even grater number of people who are just looking at this whole thing from the sideline, thinking "God, what a waste".
 
Like the "Eich incident", this has just contributed to make me more skeptical towards a cause I would definitely lend my unconditional support to - namely, equal treatment, be it of gender or any other differentiating aspect.
 
Because, once again, I don't care what is moving a persecuting mob. At the end of the day, it's still a persecuting mob.
 
Actually, I'm just about to finish, so I believe it's a good time to let Godwin in to make an argumentum ad Nazium (Nauziem, maybe?) - I see absolutely no difference between a mob that's going to indiscriminately shoot Jews, calling them all sub-humans, and a mob that's going to indiscriminately shoot Germans, calling them all Nazis. I, for one, am glad we had the "Nuremberg trials", not the "Nuremberg massacres".
 
So, if all you want is to stoop down to the lowest level of what you have elected as your "opposing tribe", I'll just stand by the sideline, watching, and taking notes. Those are notes I'll reread and evaluate the next time I'm called to vote, either with my ballot or my wallet.
 

Sunday, 24 August 2014

Productivity - Cakes and lies

On my "C++ and Ruby - Side by side" post, I mentioned having a natural affinity towards a programming language. Today, I'll elaborate on that, using something I've worked on recently to illustrate.
 
Important note: Nothing I write below is an indictment/endorsement for this or that language. What I'm going for is the exact opposite - whenever you read a book/article about a language (any language) that promises greater productivity just by switching to that language, you should be suspicious.
 

The problem

I've never liked the way IDEs (in my case, Visual Studio/Qt Creator) define their project structures, and I've been trying to setup my own structure, with the goal of supporting multiple tools.
 
Unfortunately, the tools prefer their own particular structure arrangements, and aren't very cooperative to anything that strays from their comfort zone.
 
So, in the end, I've settled for a two-step procedure - I create the projects using the IDEs and then I run a process I created to move the project to my structure. The moving part is easy, and I could do it manually. The tinkering with the project settings files is more tricky, and is what ended up giving me the final push to look at automating the whole thing.
 
I've had three attempts at creating this process.
 

Perl

The first version of this process was a Perl script. Why Perl? Well, because... scripting language... well-suited to small hacks... higher productivity... y'know?
 
It was easy to create a linear script. However, a few weeks later, when I wanted to introduce some changes, it was also easy to get lost navigating around said script, trying to get a grasp of what I had done.
 
So, I went through several redesign/refactor iterations, trying to move from a linear script to something a bit more structured.
 
I finally settled on a version that lasted several months, with Modern::Perl and Moose as foundations. As I used this script, I became aware of several weaknesses in my original design. So, a few weeks ago, I decided to review my design, and proceeded to change the script. And, again, I was lost. Even more so than in the linear script, in fact.
 
I've had to review all the scripts/modules I created in order to understand again what I had done at that time (docs? Come on, it's a simple script to create some directories, copy a few files, edit a couple of those, and init git). And, since I had to do that, I've decided to have another go at it, but this time in...
 

Ruby

Why Ruby? Well, because... scripting language... well-suited to small hacks... higher productivity... y'know?
 
It began well. I removed some syntactic quirks, especially where OO-ness was concerned. Strange as it may seem, a clean language presents a greater potential for a clear design. Maybe it's just me, maybe I'm easily distracted by these syntactic quirks, which I admit should not be quite so important.
 
Long story short, it began well, but... I've started having growing difficulties to implement my design. I've finally decided to try...
 

C++

Yes, predictable, I know...
 
Why? Because I've decided that maybe "lower productivity" was worth a shot.
 
And it went smoothly. Not "easily", not "simply"; but definitely smoothly. I've actually finished the process with the design I wanted. I can actually navigate around the code, easily finding what I'm looking for.
 
What was I aiming at? Here, look at my "main()":
 
void Run(int argc, char *argv[])
{
    AppOptions<ConfigProjectOptions>
        ao{argc, argv, "Opções ProjectConfig"};
 
    if (ao.HaveShownHelp())
    {
        return;
    }
 
    ConfigProjectOptions const& opt = ao.GetOptions();
    string project_name = opt.GetProjectName();
 
    // All objects are validated on construction.
    ProjectDirectory prj_dir(PROJ_PRJ, project_name, 
        STRUCT_PRJ, SHOULD_NOT_EXIST);
    ProjectDirectory bld_dir(PROJ_BLD, project_name, 
        STRUCT_BLD, SHOULD_NOT_EXIST);
    ProjectDirectory stg_dir(PROJ_STG, project_name, 
        STRUCT_STG, SHOULD_EXIST);
 
    Project<QtcStgValidator, QtcCopier, QtcProjectConfigUpdater>
        qtc_project{project_name, prj_dir, bld_dir, stg_dir};
    Project<MsvcStgValidator, MsvcCopier, MsvcProjectConfigUpdater>
        msvc_project(project_name, prj_dir, bld_dir, stg_dir);
 
    // Everything is valid, get user confirmation.
    if (!UserConfirms(project_name))
    {
        return;
    }
 
    prj_dir.CreateStructure();
    bld_dir.CreateStructure();
 
    if (opt.WantGit())
    {
        ConfigureGit(prj_dir.GetProjectHomeDir());
    }
 
    if (opt.IsQtcProject())
    {
        qtc_project.Copy();
    }
 
    if (opt.IsMsvcProject())
    {
        msvc_project.Copy();
    }
}

This is what I was after all along, but was unable to achieve either with Perl or Ruby. It's as clean as it gets, with the main classes clearly identified, based on the operations that I need to do, and with support classes implementing policies that actually take care of the different ways things are done.
 
The code itself is quite simple (this is a trivial program, after all), and you can find it here.
 
I probably could have achieved the same with Ruby, but this is what I'm talking about when I say "natural affinity". With C++, this code structure flowed naturally; with Ruby, not so much.
 

 What's all this about, then?

I'm repeating myself, but I'll say it anyway.
 
Don't trust productivity promises at face value, especially for quick hacks/trivial programs, where everyone says shell/scripting languages are the best choice. Sometimes, the cake is actually a lie.
 
If all you want is to get a count of particular string/regex on a log file, grep is the way to go. But say you need to do some manipulation on the results - e.g., the customer finds out he doesn't need a simple total count, but rather a list of totals for each key (e.g., customer ID). Suddenly, you're reading man pages/docs and searching the web, finding awk/perl/whatever "solutions" that don't quite give you what you want; so, you fiddle with those solutions and read some more man pages/docs.
 
And, then, you look at the clock, see how much time has passed and say - if I had fired up MS Access, I'd probably have written a little VBA, finished this and moved on (yes, "MS Access" and "VBA" are just examples, not endorsements).
 
At the end of the day, just because it's the best option for someone else, doesn't mean it's the best option for you.