Thinking in Pandas: How to Use the Python Data Analysis Library the Right Way

© Hannah Stepanek 2020

H. StepanekThinking in Pandashttps://doi.org/10.1007/978-1-4842-5839-2_8

8. Performance Improvements Beyond pandas

Hannah Stepanek¹ 

(1)

Portland, OR, USA

 

You may have heard another pandas user mention using eval and query to speed up evaluation of expressions in pandas. While use of these functions can speed up evaluation of expressions, it cannot do it without the help of a very important library: NumExpr. Use of these functions without installing NumExpr can actually cause a performance hit. In order to understand how NumExpr is able to speed up calculations however, we need to take a deep dive into the architecture of a computer.

Computer architecture

CPUs are broken up into multiple cores where each core has a dedicated cache. ...