LDV

Light-weight database virtualization (LDV) is a novel approach for sharing and repeating applications that use a relational database. LDV monitors a DB application to create a repeatability package that encapsulates the application and its dependencies (input files, binaries, and libraries) as well as the necessary and relevant data from the database required for successful repetition. LDV relies on data provenance to determine which database tuples and input files are relevant. While monitoring an application to create a package we incrementally construct an execution trace (provenance graph), that records dependencies across OS and DB boundaries. Such a package can be shared and reexecuted on any compatible machine without requiring installation of dependencies (e.g., database system or libraries) and without having to manually create and setup a database.

LDV is available as open-source software at http://github.com/IITDBGroup/ldv.git

Monitoring Applications

To use our LDV system, the user monitors an execution by prefixing the command starting the application with ldv-audit: ldv-audit run-dbapp The figure below shows how LDV monitors the execution of the user's application and its interactions with the OS and DB system. By intercepting system calls such as file operations and process creation as well as SQL statements send to the DB we incrementally build what we call an execution trace - a provenance graph that records both OS and DB operations and data dependencies. In addition to creating the execution traces and including it in the reproducibility package for the application, LDV also copies accessed files and database tuples into the package. We support two options for shipping the database. The server-include packaging option includes a DB server and the relevant DB slice in the package. The server-excluded packaging option captures the results to queries issued by the application and stores these query results in the package.

Sharing and Reexecution

To replay the execution of an application stored in a package, without any installation or configuration, the user issues for a shared package: ldv-exec run-dbapp Before starting the actual application, LDV will start-up the database server included in the package, create the schema of the application, and load the DB slice. During application execution we reroute SQL queries to the package database and file operations into the package. If the server-excluded packaging option was chosen then we replay query results included in the package from files instead of actually executing any SQL operations.

Collaborators

• Tanu Malik - DePaul University

Publications

1. 1 C ICDE LDV: Light-weight Database Virtualization
   Quan Pham, Tanu Malik, Boris Glavic and Ian Foster
   Proceedings of the 31st IEEE International Conference on Data Engineering (2015), pp. 1179–1190.
   • BibTex
   • pdf
   • slides
   • details

   @inproceedings{PM15,
     author = {Pham, Quan and Malik, Tanu and Glavic, Boris and Foster, Ian},
     booktitle = {Proceedings of the 31st IEEE International Conference on Data Engineering},
     keywords = {Provenance;LDV;Repeatability},
     pages = {1179-1190},
     pdfurl = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PM15.pdf},
     projects = {LDV},
     slideurl = {http://www.slideshare.net/lordPretzel/icde-ldv},
     title = {LDV: Light-weight Database Virtualization},
     venueshort = {ICDE},
     year = {2015},
     bdsk-url-1 = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PM15.pdf}
   }
   

   
   
   details
2. 2 J PVLDB Sharing and Reproducing Database Applications
   Quan Pham, Richard Whaling, Boris Glavic and Tanu Malik
   Proceedings of the VLDB Endowment (Demonstration Track). 8, 12 (2015) , 1988–1999.
   • BibTex
   • pdf
   • details

   @article{PW15,
     author = {Pham, Quan and Whaling, Richard and Glavic, Boris and Malik, Tanu},
     journal = {Proceedings of the VLDB Endowment (Demonstration Track)},
     keywords = {Provenance;LDV;Repeatability;Database Virtualization},
     number = {12},
     pages = {1988 - 1999},
     pdfurl = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PW15.pdf},
     projects = {LDV},
     title = {{Sharing and Reproducing Database Applications}},
     venueshort = {PVLDB},
     volume = {8},
     year = {2015},
     bdsk-url-1 = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PW15.pdf}
   }
   

   
   
   details
3. 3 W TaPP Making Database Applications Shareable
   Boris Glavic, Tanu Malik and Quan Pham
   Proceedings of the 7th USENIX Workshop on the Theory and Practice of Provenance (Poster) (2015).
   • BibTex
   • pdf
   • details

   @inproceedings{GM15,
     author = {Glavic, Boris and Malik, Tanu and Pham, Quan},
     booktitle = {Proceedings of the 7th USENIX Workshop on the Theory and Practice of Provenance (Poster)},
     isworkshop = {true},
     keywords = {Provenance;LDV;Repeatability;Database Virtualization},
     pdfurl = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/GM15.pdf},
     projects = {LDV},
     title = {{Making Database Applications Shareable}},
     venueshort = {TaPP},
     year = {2015},
     bdsk-url-1 = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/GM15.pdf}
   }
   

   
   
   details
4. 4 R Techreport LDV: Light-weight Database Virtualization
   Quan Pham, Tanu Malik, Boris Glavic and Ian Foster
   Technical Report #IIT/CS-DB-2014-03
   Illinois Institute of Technology.
   • BibTex
   • pdf
   • details

   @techreport{PM14,
     author = {Pham, Quan and Malik, Tanu and Glavic, Boris and Foster, Ian},
     date-added = {2014-10-08 20:07:29 +0000},
     date-modified = {2014-10-08 20:09:08 +0000},
     institution = {Illinois Institute of Technology},
     keywords = {Provenance;LDV;Repeatability},
     number = {IIT/CS-DB-2014-03},
     pdfurl = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PM14.pdf},
     projects = {LDV},
     title = {LDV: Light-weight Database Virtualization},
     venueshort = {Techreport},
     year = {2014},
     bdsk-url-1 = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/PM14.pdf}
   }
   

   
   
   details