Parallel Compilation Experiments in C++Builder and Dev C++
Thanks to lots of CPU cores available even in basic computer system we can significantly speed up compilation our projects. In a new article, you can look at techniques and experiments with building things faster in C++ Builder and a well-known free IDE DevC++.
This is an adapted guest post by Eli M and supported by Embarcadero.
Eli M. is an Embarcadero MVP and Growth hacker, serial entrepreneur, and full-stack developer of a variety of projects involving tens of millions of visitors. Jack of all trades software developer for the web, mobile, and desktop. Core competency lies in faster time to market by modifying existing technology to get the job done now instead of later. See his articles @Embarcadero Blog
RAD Studio is made up of Delphi and C++Builder. On the Delphi side, the Object Pascal compiler is a single-pass compiler, and the compiler itself is not a parallel compiler. Still, when compiling multiple projects in parallel, it was able to compile 1 billion lines of Object Pascal code in 5 minutes on the AMD Ryzen 9 5950x 16 core machine. I wanted to see if something similar was possible with C++.
This post is part of our modern hardware series, where we explore the massive productivity gains that can be achieved on some of the fastest CPUs available at the time of this writing in early 2021.
Just how much is 1 billion lines of code? Take a look:
Microsoft Office is around 25 million code lines, while Max OS X Tiger is 56 mln LOC.
To get to 1 billion, we need a lot of such projects. Only Google source code is estimated to reach 2 bln LOC!
C++Builder has several different compilers, including the classic Borland compiler and modern Clang based compilers for a number of platforms. Additionally, Embarcadero sponsors the open source Dev-C++, which has the TDM-GCC 9.2.0 compiler bundled with it. GCC 9.2.0 comes with MAKE which supports handles parallel compilation through it’s
-j (Jobs) command-line switch. C++Builder has an add-on called TwineCompile that brings parallel compilation to C++Builder. Both C++Builder and Dev-C++ are built with Delphi.
During my investigations so far, TwineCompile seems to offer more functionality than MAKE Jobs because TwineCompile supports background compilation and some other productivity-enhancing features. Dev-C++ is an excellent native C++ IDE for Windows development. Then C++Builder turns the productivity up to the max with its visual designer, powerful built-in VCL RTL, and enhanced parallel compilation features. Additionally, they are based around the different C++ compilers, so it is not entirely a direct comparison, and they compliment each other.
-j make Option to Dev-C++
At the beginning of this quest, Dev-C++ did not support the -j MAKE flag, so that was the first task to complete. I was able to update Dev-C++ and release the new v6.3 version with the parallel compilation -j built-in as an option now. It is also on by default for release builds, which should greatly reduce compile times for everyone using Dev-C++. This took a few days to implement and get the new v6.3 version released. Here are the release notes for Dev-C++ @Github v6.3:
Version 6.3 – 30 January 2021
- Added: Parallel compilation enabled by default for release builds through MAKE Jobs.
- Added: 3 Buttons for setting up custom shell command line tabs.
- Updated: Code completion and menus for dark themes.
- Updated: CTRL-TAB editor tab selection wrapping.
- Fixed: Make clean file deletion issue.
- Fixed: Status bar not showing all text.
- Fixed: Debug/CPU Window hex column issue.
- Fixed: Closing tabs in editor side by side view.
Here are the results that Eli M. got for his tests on a powerful Ryzen machine:
And for a 4-core system:
To see all results, detailed investigation and setup, see the following blog posts @Embarcadero Blogs:
- Compile 1 Million Lines Of C++ In ~2 Minutes With A 16 Core Ryzen 9 5950x
- Threadripper 3990X: The Quest To Compile 1 BILLION Lines Of C++ On 64 Cores
TwineCompile speeds up C++Builder compile time significantly. To test this, we got two large open-source C++ libraries which build with C++Builder: Xerces and SDL 2. Xerces is an XML library, and SDL a very well-known open-source windowing and input library, often used for games. They’re both significant test cases since they’re decent sized large C++ codebases.
- In-IDE, without TwineCompile, Xerces took 5 minutes 19 seconds to build. With TwineCompile, Xerces took 51 seconds. That’s a 6x speedup
- In-IDE, without TwineCompile, SDL 2 took 2 minutes 10 seconds to build. With TwineCompile, SDL 2 took 21 seconds. That’s a 6.2x speedup
The tests were performed on Intel i7-3930K processor (6 cores with hyperthreading.
Those numbers are impressive!
Thanks to smart techniques and caching, you can get an almost linear performance increase to the number of cores available in your system.
What’s best is that you can check this compilation model also in a completely free IDE/compiler DevC++. Here’s the Github page of this handy tool: Dev-C++ @Github.
If you want to learn more about the compilers and various options to make your projects compiling faster, check out the Embarcadero blogs: Embarcadero Blog