BUbiNG

Paolo Boldi, Andrea Marino, Massimo Santini, Sebastiano Vigna

2014 · 77 citations · 20 references

Concepts

Abstract

Although web crawlers have been around for twenty years by now, there is virtually no freely available, open-source crawling software that guarantees high throughput, overcomes the limits of single-machine tools and at the same time scales linearly with the amount of resources available. This paper aims at filling this gap.

References

20