At the March 2009 Boston MySQL User Group meeting, Jacob Nikom of MITs Lincoln Laboratory presented Optimizing Concurrent Storage and Retrieval Operations for Real-Time Surveillance Applications. In the middle of the talk, Jacob said he sometimes calls what he did in this application as real-time data warehousing, which was so accurate I decided to give that title to this blog post.
The slides can be downloaded in PDF format (1.3 Mb) at http://www.technocation.org/files/doc/Concurrent_database_performance_02.pdf.
This talk discussed how to do real-time retrieval operations while doing concurrent high volume insertion, including:
* How to keep up with 1.5 Mb/second per server incoming data stream
*
* server hardware comparison between a multi-core AMD Opteron and a multi core Intel Xeon
* MySQL/Postgres comparison
* schema design
* design of the storage/retrieval benchmark
*
[]