kdb+ is a time-series, in-memory, columnar database that has been used in the financial industry for almost 20 years. It is one of the fastest database platforms available for performing large-scale data mining, but one that is not as well-known as other NoSQL tools due to the fact that it has been used almost exclusively by hedge funds and investment banks for most of its existence. In particular, due to its speed and low overhead in processing vast amounts of data, it is used by algorithmic trading desks that engage in high-frequency trading.
With kdb+, it is fairly simple to analyze tens of millions and even hundreds of millions of records on a laptop. The main constraints would be at a hardware level - such as the amount ...