I intend to work on the scrapy benchmarking suite project. I would like to create benchmarks that reflect real-life cases as well as component-specific micro benchmarks.--profile uses cProfile for profiling and the feature is currently undocumented. I intend to investigate other profilers to be used with scrapy and try to improve the performance of the spiders based on the results.

Student

Parth-Vader

Mentors

  • Konstantin Lopukhin
  • Paul Tremberth
close

2017