Projects per year
Abstract
Many frameworks exist for programmers to develop and deploy
Big Data applications such as Hadoop Map/Reduce and Apache Spark. However, very
little debugging support is currently provided in those frameworks. When an
error occurs, developers are lost in trying to understand what has happened
from the information provided in log files. Recently, new solutions allow
developers to record & replay the application execution, but replaying is
not always affordable when hours of computation need to be re-executed. In this
paper, we present an online approach that allows developers to debug Big Data
applications in isolation by moving the debugging session to an external
process when a halting point is reached. We introduce IDRA MR , our prototype
implementation in Pharo. IDRA MR centralizes the debugging of parallel
applications by introducing novel debugging concepts, such as composite debugging
events, and the ability to dynamically update both the code of the debugged
application and the same configuration of the running framework. We validate
our approach by debugging both application and configuration failures for two
driving scenarios. The scenarios are implemented and executed using Port, our
Map/Reduce framework for Pharo, also introduced in this paper.
Original language | English |
---|---|
Awarding Institution |
|
Supervisors/Advisors |
|
Award date | 3 May 2022 |
Publication status | Published - 2022 |
Fingerprint
Dive into the research topics of 'A live debugging approach for big data processing applications'. Together they form a unique fingerprint.Projects
- 1 Active
-
SRP52: SRP-Onderzoekszwaartepunt: Foundations for Reliable Multi-Paradigm Network-Centric Programming
De Meuter, W., De Roover, C. & Gonzalez Boix, E.
1/03/19 → 29/02/28
Project: Fundamental