I found this post today on how to architect near real-time systems for risk analysis. The problem seem similar to recommendation engines which uses a lot of history data that rarely change, and only little data which change frequently.