Mozilla Firefox 35: Profile Guided Optimization

While inspecting the Firefox SlackBuild, I noticed it had an elif statement dedicated to PGO (Profile Guided Optimization):

if [ "$ARCH" = "x86_64" ]; then
  PGO=${PGO:-yes}
else
  PGO=${PGO:-no}
fi

This feature is mentioned in the SlackBuild bundled w/ the 24.1.0 extended support release (i.e. the stock 14.1 package). I decided to investigate and see what this fuss is about.

From Mozilla's FAQ:

"The Mozilla build system contains support for building with Profile-Guided Optimization (PGO) with GCC 4 or newer and Microsoft Visual C++ 2005 (Professional Edition) or newer. A PGO build consists of two passes: a first pass to build instrumented binaries, then a second pass to re-build optimized binaries using profile information gleaned from running the instrumented binaries. The Mozilla build system will run both passes for you, as well as a profile generation script in between."

From Wikipedia's page on PGO:

"Optimization techniques based on analysis of the source code alone are based on general ideas as to possible improvements, often applied without much worry over whether or not the code section was going to be executed frequently though also recognizing that code within looping statements is worth extra attention.

The first high-level compiler, introduced as the FORTRAN Automatic Coding System in 1957, broke the code into blocks and devised a table of the frequency each block is executed via a simulated execution of the code in a Monte Carlo fashion in which the outcome of conditional transfers (as via IF-type statements) is determined by a random number generator suitably weighted by whatever FREQUENCY statements were provided by the programmer. ^[3]

Rather than programmer-supplied frequency information, profile-guided optimization uses the results of profiling test runs of the instrumented program to optimize the final generated code.^[4][5] The compiler is used to access data from a sample run of the program across a representative input set. The results indicate which areas of the program are executed more frequently, and which areas are executed less frequently. All optimizations benefit from profile-guided feedback because they are less reliant on heuristics when making compilation decisions. The caveat, however, is that the sample of data fed to the program during the profiling stage must be statistically representative of the typical usage scenarios; otherwise, profile-guided feedback has the potential to harm the overall performance of the final build instead of improving it."

I decided to compile Mozilla Firefox 35 w/ PGO enabled. It took approximately 4 solid hours to compile along with 8GB of disk space.

I decided to run some tests of my own to determine the validity of this optimization hype. A quick Google search for the term "browser benchmark," led me to futuremark's peacekeeper browser benchmark:

 http://peacekeeper.futuremark.com/

Peacekeeper tests a variety of operations, including:

 > Rendering: manipulate DOM tree in real-time.
 > HTML5 - WebGL: MJS Matrix/Vector package.
 > HTML5 - Video: tests supported video formats.
 > HTML5 - Web Worker: tests javascripts multithreading.
 > HTML5 - Game: 2D sprite game (modified version of Crafty). Sprites increase as test progresses to increase load.
 > Canvas: HTML5 Canvas.
 > Data test suite:
 > DOM operations: the Document Object Model Javascript API.
 > Text Parsing:

(A detailed explanation of the tests ran in the benchmark suite can be found at the following link.)

The results were very revealing! 

 - Mozilla-Firefox-35.x86_64:                             3521 Points
 - Mozilla-Firefox-35.x86_64.pgo-enabled:       3724 Points

(The above links provide the detailed results for each package.)

That settles the argument: Firefox compiled w/ profile guided optimization has a measurable performance gain.

I am currently using the pgo-enabled Firefox build and will continue to do so unless it: creates a security risk, introduces regressions, or breaks future builds.

I have provided a PGO-enabled Firefox package for those of you who want to try it out and don't want to invest the time (and disk space), to compile:

 -> mozilla-firefox-35.0.pgo-x86_64-1.txz
 -> mozilla-firefox-35.0.pgo-x86_64-1.md5

I encourage you guys to try out the PGO-enabled build on your systems and run through the peacekeeper benchmark! I'm very curious to see any differences in performance! Also, I know I haven't provided a x86 (32bit), build yet, that will follow soon for both the vanilla firefox-35 and the PGO-enabled build!

*Also, I am asking my readers a huge favor: if anyone has an ftp, a repository, or server where I can upload these packages, I would greatly appreciate any input! These packages deserve a proper home!*

Diego