Visual Studio 2015, ICU, and error LNK2005

I'll begin by saying that I'm just going to ignore the fact that I haven't written anything in nearly nine months.

So...

While building ICU 56.1 with VS 2015, I was greeted with thousands of errors like this (also described here by someone who came across the same problem):

error LNK2005: "public: static bool const

std::numeric_limits<unsigned short>::is_signed"

(?is_signed@?$numeric_limits@...@std@@2_NB) already defined in

ParagraphLayout.obj

This is defined in <limits>, in a statement like this:

_STCONS(bool, is_signed, false);

Looking at the pre-processor output, we can see its actual definition:

static constexpr bool is_signed = (bool)(false);

If I understood the Standard correctly, this should be OK, and there should be no duplicate symbols during linking. So, I was still missing a logical cause for this.

The usual internet search for «ICU LNK2005» didn't bring anything useful, except for the link above.

Then, as I concentrated my search on LNK2005, I came across this post. The same mysterious behaviour, but now there was a plausible explanation, in a comment by MS's Stephan T. Lavavej, in a quoted post from an MSDN blog:

We recommend against using /Za, which is best thought of as "enable extra conformance and extra compiler bugs", because it activates rarely-used and rarely-tested codepaths. I stopped testing the STL with /Za years ago, when it broke perfectly conformant code like vector<unique_ptr<T>>.  

That compiler bug was later fixed, but I haven't found the time to go re-enable that /Za test coverage. Implementing missing features and fixing bugs affecting all users has been higher priority than supporting this discouraged and rarely-used compiler option.

So, after removing /Za from all projects in ICU's allinone VS Solution (Project Properties -> Configuration Properties -> C/C++ -> Language -> Disable Language Exceptions -> No), I was able to build it with no errors, on all configurations (x86/x64, debug/release).

Apparently, it's one of those rare cases where the error is actually in the compiler, not in the code.

Visual Studio - Getting to your debugging symbols

I've recently been reformulating my lib environment. Basically, it consists of:

• A loading zone, where I install the library sources, and run the build process. It has a folder for each lib, with each version in its own sub-folder.
• A library root folder, with sub-roots for mingw and MSVC, where I store the files resulting from the builds. Again, each lib folder has a sub-folder for each installed version.

There's a bit more to it, but it's not important for today's post.

This reformulation has gone through several iterations, and it's still a work in progress. This time, I had the following goals:

• G1 Getting some further progress on automating the process of building libraries from source, both release and debug versions. Actually, it was this requirement for debug versions of all the libs that led to my patch to OpenSSL with a debug configuration for mingw32.
• G2 Correcting some bad decisions regarding the lib folders' names, especially when it comes to version numbers. The goal is to use the same number format each lib uses for its own files, where applicable.
• G3 Clearing the landing zone after building the libs. Now, when I finish building, I run make (actually, mingw32-make or nmake) clean.

Point G3 led me on another learning experience.

The most common debug option when using MSVC seems to be /Zi, which stores the debug symbols in PDB files. As such, I haven't looked into the other options, which store debugging information in the object files themselves; I won't discuss them in this post, but if I had to hazard a guess, I'd say the behaviour is the same as the one described below.

During a debug build, the compiler/linker stores the absolute path to the PDB file on the executable files (EXE or DLL). When you fire up the debugger, as it loads these executable files, it looks up the PDB using this path, to load the symbols. If it can't find the PDB, it then goes through other steps.

On the open-source projects I've seen, I've come across two default behaviours - either the PDB files are not copied to the library folder at all (e.g., Boost); or they're copied to the folder containing the static libraries/import libraries, but not the DLLs themselves (e.g., ICU). I call these default behaviours, because there may be options for gettting a different behaviour; I've looked for these, but found none.

This, combined with point G3 above, is not-so-good news, because make clean deletes the PDB files. So, when we fire up the debugger, it cannot find the symbols for these DLLs. It never happened before because my procedure has always been:

• build release.
• clean up.
• build debug.
• don't clean up.

So, the PDBs were always available at the landing zone, i.e., the path stored on the executables. And even though some libs copied them to the lib folders, these weren't actually used when the DLLs were loaded by the debugger.

There are several options for dealing with this, including these three:

• O1 Setting up a local symbol server, which, as I understand it, is more of a local cache to store symbol files.
• O2 Adding each individual folder where PDB files are installed to the _NT_SYMBOL_PATH env variable, which MS debuggers use to locate symbol files.
• O3 Manually copying the PDB files to the DLLs folders.

I believe option O1 is the best, but I'll leave it for another iteration. For now, I'll go with option O3. Like I said, this is a work in progress, and I don't feel a particular pressure to go for the optimal solution (which I'll have to test before I adopt it), I prefer getting to a working solution faster, in order to keep my self-imposed deadline (which will end this weekend).

Of course, once you have all this worked out, you still need your debugger to load the correct DLLs. You may have other versions of those DLLs on your path; with popular libraries, like OpenSSL, this is more common than you may think.

On Qt Creator, assuring that you load the correct DLLs is very simple. On Projects mode (Ctrl + 5), you switch to the Run configuration and edit the PATH on the Run Environment. Since I don't have these libraries on the PATH, I always need to do this; if you usually have your debug libraries on the PATH, you don't need to edit anything.

Visual Studio is a different sort of creature. A larger sort of creature, where everything usually takes a bit more work to find.

I knew it was on Project Properties (Alt + F7). At first, I thought it was on Configuration Properties -> VC++ Directories -> Executable Directories. When I hit help, it took me here, where we can read: "Directories in which to search for executable files. Corresponds to the PATH environment variable". Fine, that's just what we need. Somewhat later, and after a moderate amount of gnashing of the dental (not mental, mind you) persuasion, I noticed that the description on the project Property Pages was a wee-bit more complete, namely "Directories in which to search for executable files while building a VC++ project". Ah... right... building the project... as in, "not running the executable".

Then, I turned to the next obvious choice, Configuration Properties -> Debugging -> Environment. This time, I read the description before going for the help button. It is quite helpful, it says "Specifies the environment for the debugee, or variables to merge with existing environment". After reading this, I knew exactly what to do. Which was hitting the help button, and hoping the help page for this option was more useful.

Fortunately, it was. We control the PATH here, using something like this: PATH=E:\Dev\lib\msvc\openssl\openssl1_0_1j\debug\bin;E:\Dev\lib\msvc\icu\icu54_1\debug\bin;%PATH%.

And, finally, after some gnashing of the mental persuasion, I can say that Visual Studio's debugger and me are finally getting along just fine.

As they say, until next time... when I turn this into reusable project configurations, instead of having to specify these manually on every project.

Logger wrapper concluded... for now

The result of this latest redesign is ready, and you can get it here.

lib_logger.h has grown quite a bit. In order to properly take advantage of __FILE__ and __LINE__ I needed to go with macros. Lots of macros, but the work in creating all these macros was softened by a little bit of perl scripting.

Note: I considered using __func__, as well, but since POCO's Logger doesn't support it, I'd have to manage that in my own Logger. However, I believe that my Logger's design goals don't include this kind of management; it's a bridge to an actual implementation, it should own nothing, and it should impose little to no unwanted weight on the client code. So, I've decided that if the client code needs __func__, it'll have to take care of it. __FILE__ and __LINE__ should be enough to follow a trail, if needed.

Another reason for this macro growth was that I wanted this to work with both GCC and MSVC. There is a difference between LoggerBridge<bool condition, etc> and the LoggerBridge<false, etc> specialization - the latter has no template functions. This poses no problem with GCC; since template functions have default template parameters, the call is similar for both template and non-template functions. On MSVC, however, template functions don't have default template parameters. Therefore, the call is object.function<type>() for one and object.function() for the other.

My first instinct was te change the specialization, changing its functions to template functions. But, after considering it further, I decided not to. Default template parameters for template functions are in the Standard, and (hopefully) sooner or later will also be in MSVC. Until then, the MSVC code will have more preprocessor "magic".

I've tested this in Qt Creator and MSVC, with my Logger (wrapper around POCO's Logger) and LoggerCout (minimum interface implementation), with values of 1 and 0 for WANT_LOGGING. All went as expected.

So, when all is said and done, what are the requirements for a client app that wants to replace the default Logger? Look at class LoggerCout for the minimum interface:

class LoggerCout

{

public:

    explicit LoggerCout(std::string const&, char const* = nullptr);

    static void CreateLogger(std::string const&, std::string const&);

    bool IsCritical() const;

    bool IsDebug() const;

    bool IsError() const;

    bool IsFatal() const;

    bool IsInformation() const;

    bool IsNotice() const;

    bool IsTrace() const;

    bool IsWarning() const;

    void Critical(std::string const& msg);

    void Debug(std::string const& msg);

    void Error(std::string const& msg);

    void Fatal(std::string const& msg);

    void Information(std::string const& msg);

    void Notice(std::string const& msg);

    void Trace(std::string const& msg);

    void Warning(std::string const& msg);

};

I don't think that's unreasonable. Of course, any other functions you need will have to be added to LoggerBridge manually. Since I've started this redesign, I've seen Herb Sutter's 2012 concurrency presentation, and was intrigued by the wrapper design he presented, especially the part about the wrapper being completely oblivious to the wrapee's interface. I'll certainly investigate if I can adapt it here, although the absence of inlining (via the use of functors/function pointers) may make it unsuitable. Anyway, for now, this is what we have.

There's one final question I'll need to answer, and that's what I'll be working on next: How does this affect performance? I'll prepare some testing with the following scenarios:

• No logging
• Logging with this solution
• Bypassing this solution and logging directly with POCO's Logger

As variations, some runs will have trivial processing, where logging should take up most of the work, and in others I'll simulate some heavy work, probably by use of sleep(). And I'll also test the use of POCO's AsyncChannel. This will give me some idea of what to expect.

Yep, I know, I finish a lot of these posts with a "Next, I'll do such and such". The truth is there's always work to be done. I like it that way.

Logger - SFINAE solution for implementation selection

Before we continue, a quick recap:

• We have some reusable code (a lib).
• We want to add logging to it.
• We want to allow the client app to replace the logger implementation.

In order to achieve the goal stated in the last point, we defined a minimum interface that the client app's logger will have to implement, and we'll now introduce a mechanism to make our lib decay into that interface, if the client app's logger doesn't implement our optimal interface.

As stated last time, we have this (slightly modified):

template<typename T, typename RESULT, 

    typename ARG1, typename ARG2, typename ARG3>

class DoesDebug

{

    template <typename U, RESULT (U::*)(ARG1, ARG2, ARG3)> struct Check;

    template <typename U> static char checkFn(Check<U, &U::Debug> *);

    template <typename U> static int checkFn(...);

public:

    enum { Exist = sizeof(checkFn<T>(0)) == sizeof(char) };

};

This will be our basic workhorse, and we'll need one for each member function.

We'll put it to work with this:

DoesDebug<Logger, void, std::string const&, char const*, int>::Exist

Upon finding this expression, the compiler will generate DoesDebug like this:

• typename T = Logger
• typename RESULT = void
• typename ARG1 = std::string const&
• typename ARG2 = char const*
• typename ARG3 = int

Now, the compiler must calculate DoesDebug<etc...>::Exist. That means, resolving checkFn<Logger>(0).

There are two candidates for this:

char checkFn(Check<U, &U::Debug> *);

int checkFn(...);

The second candidate will always be the last option for the compiler, because of its "..." argument. As for the first, it must instantiate the Check struct, like this:

template <Logger, void (Logger::*)(std::string const&, 

   char const*, int)> struct Check;

The compiler will only be able to instantiate this if &Logger::Debug can be matched with this member function pointer: void (Logger::*)(string const&, char const*, int). So, there must be an overload of Logger::Debug() that has this signature. If that is the case, the compiler successfully instantiates Check, which means sizeof(checkFn<T>(0)) is char, because that's the return type of the first overload of CheckFn; that means Exist is true, because:

Exist = sizeof(char) == sizeof(char)

On the other hand, if there is no Logger::Debug() with the required signature, the compiler can't instantiate struct Check, which means it can't instantiate the first checkFn overload, and it'll go with the second. This means sizeof(checkFn<T>(0)) is int, and Exist is false.

Hence, we have a way to check whether our lib's optimal interface exists. What we need now is a way to use it.

template <bool condition, typename LoggerImpl>

class LoggerBridge

{

public:

...

    void Debug(std::string const& msg);

    template <typename LI = LoggerImpl>

    typename std::enable_if<DoesDebug<LI, void, std::string const&, 

        char const*, int>::Exist, void>::type

    Debug(std::string const&, char const*, int);

    template <typename LI = LoggerImpl>

    typename std::enable_if<!DoesDebug<LI, void, std::string const&, 

        char const*, int>::Exist, void>::type

    Debug(std::string const&, char const*, int);

...

private:
    LoggerImpl l;
};

We'll use two identical versions of Debug(string const&, char const*, int), but using enable_if with opposed conditions (DoesDebug<> and !DoesDebug<>) ensures only one will be generated by the compiler.

These functions then become:

template <bool condition, typename T> template <typename LI>

typename std::enable_if<DoesDebug<LI, void, std::string const&,  

    char const*, int>::Exist, void>::type

LoggerBridge<condition, T>::Debug(const std::string& msg, char const* file, int line)

{

    l.Debug(msg, file, line);

}

template <bool condition, typename T> template <typename LI>

typename std::enable_if<!DoesDebug<LI, void, std::string const&, 

    char const*, int>::Exist, void>::type

LoggerBridge<condition, T>::Debug(const std::string& msg, char const*, int)

{

    l.Debug(msg);

}

So, when DoesDebug is true, i.e., when the method LoggerImpl::Debug(string const&, char const*, int) exists, the compiler will use the first function; otherwise, it uses the second, which logs the message and ignores the remaining arguments. How? The template enable_if only has a type member when the condition is true. So, when the condition is false, type does not exist, which means the compiler excludes it from overload selection.

You'll notice the functions became template functions. My first version didn't work, and after some searching, I arrived here, where Johannes Schaub - litb answer explained why:

That's because when the class template is instantiated (which happens when you create an object of type Y<int> among other cases), it instantiates all its member declarations (not necessarily their definitions/bodies!).
(...)
You need to make the member templates' enable_if depend on a parameter of the member template itself.

So, the problem is that when the Debug() functions weren't templated, the compiler instantiated all of them when LoggerBridge was instantiated. That means both versions of Debug(string const& msg, char const* file, int line) were instantiated, and GCC complained that one could not be overloaded with the other. So, I changed the member functions to templates, in order to do this: "make the member templates' enable_if depend on a parameter of the member template itself".

After getting it to work with GCC, it was time to move over to MSVC. And solve some more problems, obviously. Actually, it was just one problem - warning/error C4519 default template arguments are only allowed on a class template, because of this:

Default template arguments for function templates     No     No

The two Nos refer to VC10 and VC11. So, I gathered, if I can't use default template arguments, I'll have to use them explicitly:

#ifdef _MSC_VER 

    template <typename LI>

#else

    template <typename LI = LoggerImpl>

#endif

    typename std::enable_if<DoesDebug<LI, void, std::string const&,

        char const*, int>::Exist, void>::type

    Debug(std::string const&, char const*, int); 

And, on the call site:

#ifdef _MSC_VER 

    t.Debug<Logger>("String", __FILE__, __LINE__);

#else

    t.Debug("String", __FILE__, __LINE__);

#endif

And there you go. I'm now checking how to hide this from the user, but here I'll probably cop out and use a #define. I'll use macros for creating the logger and calling the methods, anyway; there's no way I'm going to manually write all those __FILE__s and __LINE__s.

A few closing notes.

• As I said, I'm adapting my Logger bare-bones example for this, and one thing I've already realized is the amount of boilerplate code is huge. It doesn't really bother me that much, because it's work I'll only do once. But it's still a lot more "code" than I expected.

• All of the above may be obvious to anyone experienced in C++. However, what you read here reflects my first serious contact with templates, the previous contacts being the usual Oh-templates-are-very-useful-look-how-easily-you-can-create-a-container. And this was definitely my first contact with SFINAE. Two weeks ago, all I knew was I needed a way to select function implementations at compile time and that my cursory observation of enable_if, many months ago, showed it could be done. I'm infinitely grateful to all the folks throughout the web (on StackOverflow, on other programming forums, and on their personal sites/blogs) for taking the time to explain these concepts.

• This solution seems better than what I have here, as far as boilerplate is concerned, but I didn't understand it completely. It's on my study list.

Developing with Qt on the MSVC Express IDE

You can use Qt Creator with the MSVC compiler. However, you may prefer to use the MSVC IDE, instead. This is harder than it sounds, because Qt's build process may have additional compile steps, such as invoking uic or moc; and, by default, MSVC isn't aware of these steps. There is a Qt plugin, but it doesn't work in MSVC Express. So, if you're using MSVC Express, you need a different solution.

Googling provides several answers for this. The one I found more often was using qmake -tp vc to generate a .vcxproj file out of a .pro file. Works like a charm, and you're guaranteed to get a flawless build on MSVC with no extra work. Before running this command, just make sure your PATH includes qmake, set your QMAKESPEC correctly (if you're in doubt about the correct value for your system, check the mkspecs folder in your Qt installation), and... well, you'll have some relative called Bob, I guess.

This will also set all the required include and lib folders automatically; however, it won't set the runtime path, so before you try debugging your app from the MSVC IDE, go to your project's Property Pages -> Configuration Properties -> Debugging -> Environment and set the PATH to include all the required DLLs.

Running qmake like this takes a snapshot of the .pro file and creates a similar .vcxproj file, which means that further changes to your .pro file will require running qmake again. This will generate a new .vcxproj file (amazing, hey?), so you'd better a) have a good merge tool; or b) don't plan on making any changes to the .vcxproj file.

Why is this a problem? Well, even though we're using MSVC, we'll still have to work in Qt Creator, because: 1) Working in Qt Designer is a lot easier than creating .ui files by hand; 2) adding new forms to a Qt Creator's project also sets up the corresponding C++ files; or 3) when using Designer's tools for setting up signals/slots, you must have Qt's project open, if you just double-click the .ui file in your MSVC project, it opens the corresponding form in Qt Designer, but there's some things you won't be able to do (like using the Go to slot... option) because your project isn't open in Qt Creator (i.e., Qt Creator is editing a form, but doesn't know where are the corresponding C++ files, where the code must go). So, using MSVC as your IDE doesn't mean you'll never need to fire Qt Creator again.

We could create our MSVC project like this and then, whenever we added something to our .pro file, we'd manually add it to our MSVC project (and set its Item Type to Custom Build Tool and fill all the other required fields, if necessary).

Well, we could, but this wouldn't be much of a post if there wasn't some sort of alternative, right?

And we do have an alternative. I've first come across it here. Just by using what you find there (.target files for MSBuild), you'll have automatic handling of .ui and .qrc files. MOC compilation is treated on an item-by-item basis. I've looked at this, and decided to make some changes that better fit my planned workflow.

Before we start - everything written below assumes none of your own source files follows these specs: moc_*.cpp, ui_*.h, and qrc_*.cpp. The only files that follow these specs should be the files generated by moc, uic, and rcc. If that's not the case, then do not to use anything that I present/suggest on this post; doing so can not only mess up your build process in MSVC, but can also delete your source files by erroneously assuming they were generated. You've been warned.

moc compiler

My first goal is automating moc.

Note: The post linked above states that "since usually not all headers in a Qt project use the Q_OBJECT macro moc.exe would run and fail for those headers". This might have been correct at the time it was written, but not anymore. If you run moc on a file without the Q_OBJECT macro, this is what you get


>moc sometestclass.h
sometestclass.h(0): Note: No relevant classes found. No output generated.

>echo %ERRORLEVEL%
0

Contrast that with running moc on a non-existent file


>moc mainrrr.cpp
moc: mainrrr.cpp: No such file

>echo %ERRORLEVEL%
1

So, while I could just run every .h and .cpp file through moc, I prefer running just those that actually require it. So, I created a batch script that runs a grep for Q_OBJECT and uses wc -l to count the matches.


for /f "usebackq" %%a in (`grep "Q_OBJECT" %1 ^| wc -l`) do @set ret=%%a
if not %ret% == 0 echo %1 >> %2

Yes, this is on Windows; you can either install these utilities or use another alternative. The two parameters (%1 and %2) are the source file to check for Q_OBJECT and the result file where we'll write the source file's path, if a match is found.

I've then used the .targets file for uic on the linked article and adapted it to work with my script and invoke moc. So, what does it do?

• It runs on every .h and .cpp file on the project root and its sub-folders, except files that follow the moc_*.cpp spec.

• For each of these files, it runs the batch script described above.

• It reads the result file, and runs each source file on the list through moc. moc's output is placed on either the debug or the release folder.

• It adds the moc-compiled files to ClCompile.

The last step means we don't have to manually add the moc-generated files to our project. Let's look at an example. Suppose we have a mainwindow.h that looks like this:

class MainWindow : public QMainWindow
{
    Q_OBJECT

    etc...
};

We need to run mainwindow.h through moc, which will then create moc_mainwindow.cpp. However, unless moc_mainwindow.cpp is already part of our project, it won't be included in the build. And the build will fail, obviously. This would mean that, for every x file that needed moc, we would have to manually add a moc_x.cpp to our project. Not that much work, yes, but not my cup of tea, either. Especially when we can just let the build sort it out and add it automatically.

There is just one thing I haven't yet figured out - even when this target is skipped, the moc_*.cpp files are still added to the build, which doesn't make sense to me.

uic & qrc compilers

For these, I've made almost no changes to the .targets file on the linked article.

• I've placed the output files in the debug and release folders.

• I've added the .h files to ClInclude and .cpp files to ClCompile.

Bringing it all together

So, this is my suggested approach:

• Create your Qt project, and immediately run qmake -tp vc to create the MSVC project.

• Open the MSVC IDE, and remove the following files from the project: *.ui files, moc_*.cpp files, ui_*.h files, *.qrc files, qrc_*.cpp files.

• Check the remaining files properties for any Item Type that is set as Custom Build Tool - e.g., if you have a form, qmake -tp vc will turn your form's .h file (say, mainwindow.h) Item Type to Custom Build Tool and call moc on it (because your form has the Q_OBJECT macro). Change all Item Types to C/C++ header or C/C++ compiler.

At this point, you should have a project with only .h and .cpp files, and there should be no custom build steps. Naturally, if you try to build it in this state, the build will fail, because neither moc nor uic will be called. You'll need to add the rules to call those tools. Right-click the project on the Solution Explorer and select Build Customizations. Click Find Existing... and navigate to the folder where you have the 3 .targets files described above (I've called mine QtMOC.targets, QtUIC.targets, and QtRCC.targets). Add them, and enable them (i.e., check the check-boxes before their names).

If you build now, you should have a successful build.

So, how do I plan to use this setup, then?

• Changes to the form should be made in Qt Designer, opening the Qt project with Qt Creator.

• Any UI additions will require us to manually add the .h and .cpp files. There's no need to add the .ui file, it'll be picked up by our rule.

• Any new C++ source file will have to be manually added.

• Nothing else needs to be added to the MSVC project. Our rules will pick up all the necessary files and include them in the build.

This is a level of manual work I'm comfortable with, because all it entails is adding C++ files. I'm not worrying about adding any specific Qt files, those are handled automatically.

I'll be using the same folder as project root for both Qt Creator's and MSVC's projects. The generated files will go on different sub-folders, though - I'll use MSVC's default debug and release folders, and for Qt Creator, I'll call them something like QtC-<Kit>-debug and QtC-<Kit>-release.

The .targets files output messages to the build log. However, to see those messages, you have to go to Tools -> Options -> Projects and Solutions -> Build and Run and set both MSBuild project build output verbosity and MSBuild project build log file verbosity to Normal. Alternatively, you can tweak the Importance parameter of the Message tasks.

Clean

You'll notice when you run Build -> Clean Solution, the generated files remain. If you don't want that, there's a simple way to take care of it. Open your project's Property Pages, go to Configuration Properties -> General -> Extensions to Delete on Clean, and select Edit from the combo box. Add the following lines:

moc_*.cpp

ui_*.h

qrc_*.cpp

Yep, even though it says extensions, it actually uses the whole file spec, which is the intelligent thing to do, actually. I'll say it again, obvious though it may be: If any of your own files follow these specs, then you can't use this.

You don't need to worry about dealing with the debug or release folders, because MSVC runs Clean on those folders, depending on which configuration you have active - if you copy a moc_*.cpp file and put it on your project's root folder, or on any other sub-folder, it won't be deleted.

There's an alternative to this, which is to place all the generated files in one folder (e.g., generated_files), and adding something like this:

generated_files\moc_*.cpp

generated_files\ui_*.h

generated_files\qrc_*.cpp

This means you'd have to create this folder in both debug and release folders. If you wanted a single generated_files folder, under your project's root folder, you'd have to add "..\" to each of the 3 lines above.

A final note

I'm just learning this, and none of the above has been thoroughly tested. Hence, the disclaimers on the .targets files. Use this at your own risk.

You can get the files here.

Building changes

So, long time, no writing... again.

After some more testing, and a fair bit of reading, I've come to the conclusion that I should change my work environment.

I became more aware of the MinGW fracture, and its implications. I've been using mingw.org's MinGW (I'll just call it MinGW), and I had planned to have 3 environments - MinGW, MSVC, and Clang. I've decided to add mingw-32 (already added by Qt Creator 2.6.2) and mingw-64. Yes, there has been plenty of building going on (I might change my name to Bob, eventually), but, as usual, there's also been plenty of learning (this should be a hint that things went less than smoothly).

The plan was as follows: Build for mingw, mingw-32 and MSVC. Then, add mingw-64. And, finally, Clang. I've since decided I'll leave these two for a later time, for now I'll just focus on what I already have on my system.

Building... never a dull moment, heh? And I'm fortunate enough to count Mr. Murphy among my friends, so I like to think I'm given a bonus, as far as non-dull-moments are concerned.

On my first attempt(s) at building Qt 5 (with MinGW), I included WebKit. And Mr. Murphy was kind enough to provide me with a token of our aforementioned friendship (and a new learning experience). Consider this:

g++ <snip an obscene amount of -Ds and -Is> -o path\to\InspectorBackendCommands.o path\to\InspectorBackendCommands.cpp

This was where the build first failed. And the error message complained about not being able to find the file InspectorBackendCommands.cpp. More exactly, the file path\toInspectorBackendCommands.cpp. Ah, you noticed the missing separator as well? Good. Because it sent me on a lovely quest to understand what exactly could be causing this particular concatenation to fail. The fact that it failed on both a path separator and an escape character gave this theory a bit more credibility.

Naturally, after taking a look at the makefiles (and the .pro files, and the .pri files) I realized the concatenation couldn't be the cause of the problem. I decided to run the command individually, and when I pasted it on the cmd prompt, it got truncated. Which led me to suspect some sort of limit was hit by this particular g++ invocation. Which was then confirmed by this.

However, something else was going on here, because the error message didn't truncate anything. It just removed a character. Since I ran make with debug on, I knew it was creating a temporary .bat file and running it, so I recreated the .bat file, I inserted "-DWEBKIT_DO_YOU_REALLY_NEED_ALL_THIS_CRAP_TO_BUILD" into the g++ invocation and ran it. And, naturally, it was now missing some other character.

I still don't understand what exactly is going on here. It's a sort of limit, but one that apparently removes one character, instead of truncating a string. Still, as much as I like a good challenge, I've decided to let this one pass. I made an half-assed attempt at recovering from this failure - I shortened this invocation by removing some stuff I suspected wasn't necessary and reran make. It soon stopped again, on another obscenely long line, much longer than this one. I could've tried to shorten this one, too, but I didn't even know where to start.

In the end, I decided to follow this piece of advice from here:

Consider skipping qtwebkit (...). This module is quite big, takes a long time to compile and if often a source of compile errors, so it is recommend to only download it if you intend to use it.

I don't know if I intend to use it, but I know I don't intend to spend the necessary time to solve what is not, at this moment, an important problem.

So, I deleted everything, recloned the git repo, and ran init-repository with --no-webkit. And it built.

I've since found out that there's a solution for this, but it involves changing mingw's make and rebuilding it. I'm not going to do that now.

All the other builds went smoothly with both versions of MinGW. Qt posed no problems because it was already built with mingw-32.

Then, I started building with MSVC 2012 Desktop. Everything went OK (some minor problems, easily solved), until Qt. While it built, it didn't build properly, some modules were missing (e.g., webkit wasn't even showing up in the logs). I've posted a request for help here, but got no luck. Then, I got suspicious that maybe Qt's cleaning process wasn't quite working as it should, so I deleted my local repo and cloned it again.

And this time, nmake gave me an error, about webkit requiring zlib. Which was odd, because I had zlib, and I was including it in the build paths. Then, I decided to just use -qt-zlib, and everything went smoothly.

So, of all the libs I've built (openssl, zlib, libssh2, icu, boost, poco, qt), qt is definitely the more finicky, at least on Windows. But, since it's also the lib with the largest scope, I guess that's to be expected.

Next time: Testing the builds.