linux app compiled with optimizations?

Message boards : Number crunching : linux app compiled with optimizations?

To post messages, you must log in.

AuthorMessage
biodoc

Send message
Joined: 22 Apr 18
Posts: 3
Credit: 863,310
RAC: 40,546
Message 231 - Posted: 15 Jul 2018, 10:48:55 UTC

The linux app appears to be about 30% slower than the windows app when comparing task computation times.

This computer running windows completes tasks on average of 3,778 seconds

Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
Number of processors 40

http://www.acousticsathome.ru/boinc/results.php?hostid=13922&offset=0&show_names=0&state=4&appid=

My computer with same processors running linux completes tasks on average of 4,705 seconds.

Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
Number of processors 40

http://www.acousticsathome.ru/boinc/results.php?hostid=15856&offset=0&show_names=0&state=4&appid=

Also, there are only 3 computers running linux in the top 40 in the top hosts list.

I was just wondering if the linux app was compiled using any optimization flags such as described here:
http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
ID: 231 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
xii5ku

Send message
Joined: 14 Apr 18
Posts: 1
Credit: 1,683,822
RAC: 0
Message 233 - Posted: 22 Jul 2018, 12:35:19 UTC

I confirm this observation. I ran the Linux and Windows application on the very same host, which dual-boots either Linux Mint 18.3 or Windows 7 Pro SP1. Hardware: Xeon E5-2696 v4 (Broadwell-EP, 22 cores, 44 threads).

  • SSPEMDD v0.18 windows_x86_64:
    Run time: 3,503 s average, 67 s standard deviation
    CPU time: 3,494 s average, 68 s standard deviation

  • SSPEMDD v0.18 x86_64-pc-linux-gnu:
    Run time: 4,816 s average, 67 s standard deviation
    CPU time: 4,813 s average, 66 s standard deviation


I.e., run time of Windows : Linux = 58.4 min : 80.3 min = 1.00 : 1.37.

Each sample consisted of 44 tasks which were run simultaneously on the otherwise idle host.
The Linux tasks were downloaded on 15 Jul 2018, 11:15:55 UTC. Current status: 36 valid, 8 pending validation.
The Windows tasks were downloaded on 15 Jul 2018, 19:28:46 UTC. Current status: 40 valid, 4 pending validation.

ID: 233 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
fractal

Send message
Joined: 6 Apr 17
Posts: 2
Credit: 300,975
RAC: 2,544
Message 236 - Posted: 1 Aug 2018, 2:22:24 UTC

Do you plan on releasing the source code so others can optimize it?
ID: 236 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Crunch3r

Send message
Joined: 2 Apr 17
Posts: 1
Credit: 23,520
RAC: 0
Message 241 - Posted: 4 Sep 2018, 20:14:02 UTC - in response to Message 236.  

Do you plan on releasing the source code so others can optimize it?


The source code is available at https://github.com/Nauchnik/Acoustics-at-home

Feel free to take a look at it. However, the app is not the real culprit. It's rather the use of the free ALGLIB which takes most of the time (~95%, have a look at the evd_internaldlaebz function).

So you have to look at the ALGLIB source code to optimize/vectorize using SSE2 or AVX, or switch to the comercial version of ALGLIB that has support for SSE2/AVX optimized code.

Or get rid of ALGLIB in the first place and use something like GSL(free) ,Intel MKL (commercial) or other alternatives.
ID: 241 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oleg Zaikin
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 28 Mar 17
Posts: 116
Credit: 1,601,711
RAC: 0
Message 250 - Posted: 11 Nov 2018, 17:24:29 UTC - in response to Message 231.  

The linux app appears to be about 30% slower than the windows app when comparing task computation times.

I was just wondering if the linux app was compiled using any optimization flags such as described here:
http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html


I confirm that the Linux application is slower than the Windows one. The reason is that the Windows app was compiled by Visual Studio compiler, while the Linux app was compiled by GCC. The Linux app was compiled with -O3 flag, so I do not know how to optimize it better. I suppose, that GCC is not as good in compiling vector opearations as Visual Studio compiler is.
ID: 250 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oleg Zaikin
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 28 Mar 17
Posts: 116
Credit: 1,601,711
RAC: 0
Message 251 - Posted: 11 Nov 2018, 17:30:46 UTC - in response to Message 241.  


Feel free to take a look at it. However, the app is not the real culprit. It's rather the use of the free ALGLIB which takes most of the time (~95%, have a look at the evd_internaldlaebz function).

So you have to look at the ALGLIB source code to optimize/vectorize using SSE2 or AVX, or switch to the comercial version of ALGLIB that has support for SSE2/AVX optimized code.

Or get rid of ALGLIB in the first place and use something like GSL(free) ,Intel MKL (commercial) or other alternatives.


Yes, you are right, the ALGLIB functions take almost all computational resources. Yes, it seems that the main question is how a compiler deals with these functions. We tried other free implementations of required methods - they work slower. I'm going to take a look at the commercial version of ALGLIB.
ID: 251 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
[B@P] Daniel

Send message
Joined: 31 Mar 17
Posts: 10
Credit: 4,726,894
RAC: 9,401
Message 293 - Posted: 13 Dec 2018, 11:25:31 UTC
Last modified: 13 Dec 2018, 12:13:30 UTC

My past experiences with compilers are opposite, gcc usually can generate better code than msvc. I think that reason for why linux app is slower is different. You are compiling every alglib file separately, so gcc cannot inline various small utility functions from ap.cpp. Please compile all of them together and check how this affected app performance. You can do this by passing all of them to one gcc invocation, or create a new file and #include all .cpp files in it. You can also try Link Time Optimization (-flto) which has similar effect, but for this you will need some reasonably new gcc version.

Edit: you can also try to compile your cpp files together with alglib ones. This may lead to even better results, thanks to constant propagation.
ID: 293 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : linux app compiled with optimizations?