English
Related papers

Related papers: Anywhere: A Web Crawler Automation Management Inte…

200 papers

Benchmarking is commonly used in research fields, such as computer architecture design and machine learning, as a powerful paradigm for rigorously assessing, comparing, and developing novel technologies. However, the data centre networking…

Networking and Internet Architecture · Computer Science 2022-08-26 Christopher W. F. Parsonson , Joshua L. Benjamin , Georgios Zervas

Obtaining the desired dataset is still a prime challenge faced by researchers while analyzing Online Social Network (OSN) sites. Application Programming Interfaces (APIs) provided by OSN service providers for retrieving data impose several…

Social and Information Networks · Computer Science 2019-11-27 Mudasir Ahmad Wani , Nancy Agarwal , Suraiya Jabin , Syed Zeeshan Hussai

Web crawling is a critical technique for extracting online data, yet it poses challenges due to webpage diversity and anti-scraping mechanisms. This study investigates the integration of generative AI tools Claude AI (Sonnet 3.5) and…

Information Retrieval · Computer Science 2025-02-26 Chau-Jian Huang

The historical, cultural, and intellectual importance of archiving the web has been widely recognized. Today, all countries with high Internet penetration rate have established high-profile archiving initiatives to crawl and archive the…

Digital Libraries · Computer Science 2013-08-13 Zhiwu Xie , Herbert Van de Sompel , Jinyang Liu , Johann van Reenen , Ramiro Jordan

In recent years, the research community has raised serious questions about the reproducibility of scientific work. In particular, since many studies include some kind of computing work, reproducibility is also a technological challenge, not…

Software Engineering · Computer Science 2023-08-03 Lázaro Costa , Susana Barbosa , Jácome Cunha

Internet users are vulnerable to privacy attacks despite the use of encryption. Webpage fingerprinting, an attack that analyzes encrypted traffic, can identify the webpages visited by a user in a given website. Recent research works have…

Cryptography and Security · Computer Science 2022-05-31 Gargi Mitra , Prasanna Karthik Vairam , Sandip Saha , Nitin Chandrachoodan , V. Kamakoti

User interface design is a complex task that involves designers examining a wide range of options. We present Spacewalker, a tool that allows designers to rapidly search a large design space for an optimal web UI with integrated support.…

Human-Computer Interaction · Computer Science 2021-02-19 Mingyuan Zhong , Gang Li , Yang Li

We propose an architecture, Flare, that is a structured and easy way to develop applications rapidly, in a multitude of languages, which make use of online storage of data and management of users. The architecture eliminates the need for…

Networking and Internet Architecture · Computer Science 2015-05-14 Shashank Shekhar , Mohit Soni , NVSN Kalyan Chakravarthy

Collections of Web documents about specific topics are needed for many areas of current research. Focused crawling enables the creation of such collections on demand. Current focused crawlers require the user to manually specify starting…

Digital Libraries · Computer Science 2016-12-20 Gerhard Gossen , Elena Demidova , Thomas Risse

Web services are widely used in many areas via callable APIs, however, data are not always available in this way. We always need to get some data from web pages whose structure is not in order. Many developers use web data extraction…

Databases · Computer Science 2019-10-18 Naibo Wang , Zhiling Luo , Xiya Lyu , Zitong Yang , Jianwei Yin

Majority of the computer or mobile phone enthusiasts make use of the web for searching activity. Web search engines are used for the searching; The results that the search engines get are provided to it by a software module known as the Web…

Information Retrieval · Computer Science 2014-11-18 Prashant Dahiwale , M M Raghuwanshi , Latesh malik

Achieving accessibility compliance is extremely important for many government agencies and businesses who wish to improve services for their consumers. With the growing reliance on dynamic web applications many organizations are finding it…

Information Retrieval · Computer Science 2021-12-13 Trevor Bostic , Jeffrey Stanley , John Higgins , Daniel Chudnov , Justin Brunelle , Brittany Tracy

Web crawling is the problem of keeping a cache of webpages fresh, i.e., having the most recent copy available when a page is requested. This problem is usually coupled with the natural restriction that the bandwidth available to the web…

Machine Learning · Computer Science 2019-11-26 Utkarsh Upadhyay , Robert Busa-Fekete , Wojciech Kotlowski , David Pal , Balazs Szorenyi

The clear, social, and dark web have lately been identified as rich sources of valuable cyber-security information that -given the appropriate tools and methods-may be identified, crawled and subsequently leveraged to actionable…

Cryptography and Security · Computer Science 2021-09-16 Paris Koloveas , Thanasis Chantzios , Christos Tryfonopoulos , Spiros Skiadopoulos

Meeting performance and scalability requirements while delivering services is a critical issue in web applications. Recently, latency and cost of Internet-based services are encouraging the use of application-level caching to continue…

Software Engineering · Computer Science 2020-11-03 Jhonny Mertz , Ingrid Nunes

In this paper, we argue that modern functional programming languages - in particular, FSharp on the .NET platform - are well suited for the development of distributed, web and cloud applications on the Internet. We emphasize that FSharp can…

Programming Languages · Computer Science 2015-12-08 Dmitri Soshnikov

Real-time processing of data streams emanating from sensors is becoming a common task in Internet of Things scenarios. The key implementation goal consists in efficiently handling massive incoming data streams and supporting advanced data…

Databases · Computer Science 2017-05-17 Xiangnan Ren , Olivier Curé

Preventing abusive activities caused by adversaries accessing online services at a rate exceeding that expected by websites has become an ever-increasing problem. CAPTCHAs and SMS authentication are widely used to provide a solution by…

Cryptography and Security · Computer Science 2024-06-27 Kosei Akama , Yoshimichi Nakatsuka , Masaaki Sato , Keisuke Uehara

Distributed dataflow systems like Apache Spark and Apache Hadoop enable data-parallel processing of large datasets on clusters. Yet, selecting appropriate computational resources for dataflow jobs -- that neither lead to bottlenecks nor to…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-01-11 Jonathan Will , Lauritz Thamsen , Jonathan Bader , Dominik Scheinert , Odej Kao

The development of a real-time web application often starts with a feature-driven approach allowing to quickly react to users feedbacks. However, this approach poorly scales in performance. Yet, the user-base can increase by an order of…

Programming Languages · Computer Science 2015-12-23 Etienne Brodu , Stéphane Frénot , Frédéric Oblé