Remove Data Engineering Remove Engineering Culture Remove Hardware Remove Testing
article thumbnail

How to Save Time and Money by Testing Spark Locally

Xebia

Why should you think once again about testing Spark? Data Engineers were tempted by the pressure of the moment to give up on testing all together. There was no need for generating your own data; just take a percentage of production data. Shouldering blame: if something goes wrong, blame your Data Engineer!

Testing 130
article thumbnail

Article: Using Machine Learning for Fast Test Feedback to Developers and Test Suite Optimization

InfoQ Culture Methods

Software testing, especially in large scale projects, is a time intensive process. Test suites may be computationally expensive, compete with each other for available hardware, or simply be so large as to cause considerable delay until their results are available.