Quantcast

Dynamo RDTSC / Linux aio_* API questions

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Dynamo RDTSC / Linux aio_* API questions

Jeff Squyres (jsquyres)
Greetings.

1. It looks like IOTime.cpp is using the RDTSC clock for time measurement.  This isn't safe on multicore systems (e.g., if the OS moves the dynamo process from processor socket A to processor socket B, the RDTSC values are likely to be unrelated).  Is there any thought of changing the use of RDTSC to some other method?  E.g., on Linux, the clock_gettime() method can be used.

Or is processor affinity always enforced to lock dynamo in place so that consecutive RDTSC values are relevant?  I ask because it *looks* like you can disable affinity, but the RDTSC clock is still used...?  

Please feel free to tell me that I completely misunderstand the code.  :-)

2. A quick browse through the source code shows that IOCompletionQ.cpp is using the aio_* API for reads and writes.  Is there any effort going into using the Linux-native io_* API for reads and writes?  It seems to perform significantly better than the aio_* API (in RHEL 5 and 6, at least).

Thanks!

--
Jeff Squyres
[hidden email]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/


------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Iometer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iometer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dynamo RDTSC / Linux aio_* API questions

Allen, Wayne
Hi Jeff,

Thanks for pinging us regarding your questions.

1. We have migrated the Windows portion of the code away from RDTSC due to the reasons you mentioned. Linux hasn't been addressed yet.

2. You point out something that has been under discussion among us IOMeter admins for some time. We'd like to make that migration; however we haven't had the bandwidth to get to it. We'd certainly entertain others contributing code that gets it done. :)

Best Regards,

Wayne

-----Original Message-----
From: Jeff Squyres [mailto:[hidden email]]
Sent: Wednesday, July 13, 2011 12:54 PM
To: [hidden email]
Subject: [Iometer-devel] Dynamo RDTSC / Linux aio_* API questions

Greetings.

1. It looks like IOTime.cpp is using the RDTSC clock for time measurement.  This isn't safe on multicore systems (e.g., if the OS moves the dynamo process from processor socket A to processor socket B, the RDTSC values are likely to be unrelated).  Is there any thought of changing the use of RDTSC to some other method?  E.g., on Linux, the clock_gettime() method can be used.

Or is processor affinity always enforced to lock dynamo in place so that consecutive RDTSC values are relevant?  I ask because it *looks* like you can disable affinity, but the RDTSC clock is still used...?  

Please feel free to tell me that I completely misunderstand the code.  :-)

2. A quick browse through the source code shows that IOCompletionQ.cpp is using the aio_* API for reads and writes.  Is there any effort going into using the Linux-native io_* API for reads and writes?  It seems to perform significantly better than the aio_* API (in RHEL 5 and 6, at least).

Thanks!

--
Jeff Squyres
[hidden email]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/


------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Iometer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iometer-devel

------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Iometer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iometer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dynamo RDTSC / Linux aio_* API questions

Jeff Squyres (jsquyres)
On Jul 13, 2011, at 5:44 PM, Allen, Wayne wrote:

> 1. We have migrated the Windows portion of the code away from RDTSC due to the reasons you mentioned. Linux hasn't been addressed yet.

Gotcha.

> 2. You point out something that has been under discussion among us IOMeter admins for some time. We'd like to make that migration; however we haven't had the bandwidth to get to it. We'd certainly entertain others contributing code that gets it done. :)

I'm Cisco's contributor to another open source project, so I can certainly understand "patches are welcome!".  I do believe I've used that phrase a few times myself.  :-)

Sadly, I don't have the cycles to do such a port to the Linux-specific io_* API at the moment.  :-(  I was mainly asking if anyone else was doing it.

--
Jeff Squyres
[hidden email]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/


------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Iometer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iometer-devel
Loading...