InfoQ Culture Methods

APRIL 7, 2020

QCon returned to London this past March for its fourteenth year in the city, attracting over 1,600 senior developers, architects, data engineers, team leads, and CTOs. This article provides a summary of the key takeaways. By Abel Avram.

Data Engineering 80

Data Engineering Engineering Data Development 80

Toptal

DECEMBER 6, 2016

Recently, new data storage technologies have emerged. Which one is best suited for data engineering? In this article, Toptal Data Scientist Ken Hu compares three prominent storage technologies within the context of data engineering. But the question is: Which one should you choose?

Data Engineering 85

Data Engineering Storage Engineering Data 85

InfoQ Culture Methods

JANUARY 13, 2023

This article is a summary of the 2022 software trends podcast. 2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022, and what to watch in 2023.

Trends 115

Trends Software Report Innovation 115

InfoQ Culture Methods

MAY 24, 2023

GDPR compliance should be a default feature in every application that handles PII (Personally Identifiable Information). Most organizations have an impression that GDPR is a luxury feature that needs special tools to implement.

Programming 129

Programming Compliance Applications Tools 129

InfoQ Culture Methods

JANUARY 23, 2024

Generative AI will profoundly transform communication and information sharing over the next decade, but the change will be uneven across industries and roles. Organizations should empower workers to use AI augmentation thoughtfully, while building literacy on capabilities and limits.

Generative AI 60

Generative AI Industry Organization Data Engineering 60

InfoQ Culture Methods

FEBRUARY 22, 2022

The article explores optimizing test execution, saving machine resources, and reducing feedback time to developers. Test suites may be computationally expensive, compete with each other for available hardware, or simply be so large as to cause considerable delay until their results are available. By Gregor Endler, Marco Achtziger.

Machine Learning 124

Machine Learning Artificial Inteligence Testing Development 124

InfoQ Culture Methods

SEPTEMBER 1, 2023

There’s an increasing concern about the energy use and corresponding carbon emissions of generative AI models. And while the concerns may be overhyped, they still require attention, especially as generative AI becomes integrated into our modern life.

Generative AI 109

Generative AI Energy Architecture System 109

InfoQ Culture Methods

JANUARY 7, 2022

This article presents how to use data science to detect wastes and impediments, and concepts and related information that help teams to figure out the root cause of impediments they struggle to get rid of.

Software Development 109

Software Development Data Software Trends 109

CIO

OCTOBER 6, 2020

Earlier this year, Microsoft announced it would retire 15 certifications and 44 exams, including MCSA: BI Reporting, which validated data analysts' expertise using Power BI and Excel to visualize data via dashboards and reports. To read this article in full, please click here

Data 240

Data Data Engineering Report Engineering 240

InfoQ Culture Methods

JULY 17, 2020

To mix the power of the data and the importance of people to offer business intelligence is a key point nowadays. Innovation is not only about the most advanced technology, management and processes are the new era of startups' innovation. The result is not only the most imporant thing, the way you do it more important.

Agile 138

Agile Innovation Culture Business Intelligence 138

InfoQ Culture Methods

DECEMBER 28, 2020

As 2020 is coming to an end, we created this article listing some of the best posts published this year. This collection was hand-picked by nine InfoQ Editors recommending the greatest posts in their domain. It's a great piece to make sure you don't miss out on some of the InfoQ's best content.

Continuous Delivery 101

Continuous Delivery Google Cloud Data Engineering Microservices 101

InfoQ Culture Methods

FEBRUARY 25, 2021

Developing ML with agile has a few challenges that new teams coming up in the space need to be prepared for - from new roles like Data Scientists to concerns in reproducibility and dependency management. Machine learning is a powerful new tool, but how does it fit in your agile development? By Jay Palat.

Machine Learning 133

Machine Learning Artificial Inteligence Agile Development 133

InfoWorld

MARCH 8, 2024

LinkedIn has decided to open source its data management tool, OpenHouse, which it says can help data engineers and related data infrastructure teams in an enterprise to reduce their product engineering effort and decrease the time required to deploy products or applications.

Open Source 77

Open Source Tools Data Engineering Storage 77

Dzone - DevOps

OCTOBER 30, 2023

In this article, we explore the synergy between software-based architecture and the development of interoperability solutions for IoT to provide insights relevant to software developers and data engineers. To tackle this challenge head-on, software-based architectures are emerging as powerful solutions.

IoT 87

IoT Architecture Software Data Engineering 87

Harvard Business Review

MAY 10, 2024

They determined that, although data engineers and data scientists typically take on most responsibility from conception to production of AI development lifecycles, non-technical leaders can play a key role in ensuring the integration of responsible AI.

How To 104

How To Data Engineering Research Engineering 104

The Crazy Programmer

NOVEMBER 6, 2021

Big Data Engineer. Another highest-paying job skill in the IT sector is big data engineering. And as a big data engineer, you need to work around the big data sets of the applications. Not only this, but you also need to use coding skills, data warehousing, and visualizing skills. Conclusion.

Artificial Inteligence 290

Artificial Inteligence Artificial Intelligence Blockchain Software Review 290

InfoQ Culture Methods

JANUARY 19, 2022

In this podcast summary Thomas Betts, Wes Reisz, Shane Hastie, Charles Humble, Srini Penchikala, and Daniel Bryant discuss what they have seen in 2021 and speculate a little on what they hope to see in 2022. Topics explored included: hybrid working and the importance of ethics and sustainability within technology.

Technical Review 100

Technical Review Sustainability Artificial Inteligence Technology 100

InfoQ Culture Methods

JULY 28, 2020

To become a machine learning engineer, you have to interview. You have to gain relevant skills from books, courses, conferences, and projects. Include technologies, frameworks, and projects on your CV. In an interview, expect that you will be asked technical questions, insight questions, and programming questions.

Machine Learning 138

Machine Learning Artificial Inteligence Engineering How To 138

InfoQ Culture Methods

DECEMBER 6, 2019

As part of the 2019 end-of-year-summary content, this article collects together a list of recommended presentation recordings from the InfoQ editorial team. By Charles Humble, Ben Linders, Arthur Casals, Manuel Pais, Erik Costlow, Shane Hastie, Shaaron A Alvares, Srini Penchikala, Michael Redlich, Steef-Jan Wiggers, Daniel Bryant.

Data Engineering 52

Data Engineering Architecture Programming System 52

InfoQ Culture Methods

SEPTEMBER 6, 2019

Is it all about customer-centric business, or is there any data left? Can we integrate data analytics and customer empathy? This article explores how we can move towards a more customer-centric business and what information we require in order to understand the most valuable thing we have: our customer.

Analytics 65

Analytics Agile Data Big Data 65

InfoWorld

DECEMBER 4, 2023

But there was plenty other news to spark the interest of database managers, data scientists, data engineers, and developers, including new extract, transform, load (ETL) services, a new Cost Optimization Hub, and revamped enterprise pricing tier for AWS' cloud-based development tool, dubbed Amazon CodeCatalyst.

AWS 69

AWS Artificial Intelligence Artificial Inteligence Generative AI 69

InfoQ Culture Methods

FEBRUARY 16, 2023

Have you ever wondered about systems based on machine learning? In those cases, testing takes a backseat. And even if testing is done, it’s done mostly by developers itself. A tester’s role is not clearly portrayed. Testers usually struggle to understand ML-based systems and explore what contributions they can make.

Artificial Inteligence 100

Artificial Inteligence Machine Learning System Testing 100

InfoQ Culture Methods

MARCH 19, 2020

The book AI Crash Course by Hadelin de Ponteves contains a toolkit of four different AI models: Thompson Sampling, Q-Learning, Deep Q-Learning and Deep Convolutional Q-learning. It teaches the theory of these AI models and provides coding examples for solving industry cases based on these models. By Ben Linders, Hadelin de Ponteves.

Course 109

Course Software Review Industry Examples 109

InfoQ Culture Methods

MAY 4, 2020

These days AI is a big buzzword. While it rises in popularity, the controversy surrounding it flourishes as well.

Testing 130

Testing Technology Data Engineering Artificial Intelligence 130

Dzone - DevOps

FEBRUARY 7, 2023

As digital transformation reaches more industries, the number of data points generated is growing exponentially. As such, data integration strategies to collect such large volumes of data from different sources in varying formats and structures are now a primary concern for data engineering teams.

Strategy 95

Strategy Data Data Engineering Analysis 95

Perficient

MARCH 23, 2023

Databricks Streaming and Apache Flink are two popular stream processing frameworks that enable developers to build real-time data pipelines, applications and services at scale. Comparison Databricks is an integrated platform for data engineering, machine learning, data science and analytics built on top of Apache Spark.

Data 83

Data Machine Learning Artificial Inteligence Data Engineering 83

InfoQ Culture Methods

DECEMBER 30, 2019

To be able to improve features in games which are constantly evolving, the challenge will be to scale tests to be on a par with new feature development. Automated tests are vital for King to keep up testing Candy Crush, therefore they are constantly looking for new improved ways to test. By Alexander Andelkovic.

Testing 52

Testing Games Development Case Study 52

Data Virtualization

DECEMBER 23, 2021

Reading Time: 5 minutes As I embark on my new journey with Denodo and data virtualization, I am now frequently asked, “What is data virtualization?” Unless I am talking to a data engineer or a data architect (this would help explain it, if you.

Virtualization 52

Virtualization Data Data Engineering Analysis 52

Data Virtualization

DECEMBER 23, 2021

Reading Time: 5 minutes As I embark on my new journey with Denodo and data virtualization, I am now frequently asked, “What is data virtualization?” Unless I am talking to a data engineer or a data architect (this would help explain it, if you.

Virtualization 52

Virtualization Data Data Engineering Analysis 52

Exadel

OCTOBER 18, 2022

No matter the industry, a practical data science approach brings data scientists, stakeholders, and data engineers together to drive business value. The role of a data science team is to use one of three approaches, depending on the maturity of an organization. Data capability and infrastructure assessment.

Data 52

Data Data Engineering Infrastructure Industry 52

InfoQ Culture Methods

APRIL 18, 2020

At the end of 2019, InfoQ ran a survey of our readers to find out what tools, techniques, and languages they were using. This is a summary of the results. By Charles Humble.

Survey 56

Survey Tools Data Engineering Programming 56

Dataiku

AUGUST 30, 2022

This is a guest article by Sooter Saalu. Sooter is a data professional with over four years of experience working across data-centric roles. He has a working knowledge of Python, SQL, and several BI and data engineering tools.

Data Engineering 52

Data Engineering Education Engineering Data 52

CIO

SEPTEMBER 29, 2023

AI content creation Generative AI’s promise for content creation can’t be denied, and more companies are turning to generative AI to create content such as blog posts, social media posts, graphics, articles, and even videos. With generative AI, this skill is important for creating quality consumer-facing products and services.

Generative AI 357

Generative AI ChatGPT Machine Learning Artificial Inteligence 357

InfoQ Culture Methods

DECEMBER 6, 2019

We take a look back at what we saw on InfoQ in 2019, and think about what the next year might bring. By Charles Humble, Erik Costlow, Arthur Casals, Daniel Bryant, Bruno Couriol, Ben Linders, Shane Hastie.

Software 52

Software Data Engineering .Net Architecture 52

Altexsoft

FEBRUARY 11, 2023

This suggests that today, there are many companies that face the need to make their data easily accessible, cleaned up, and regularly updated. Hiring a well-skilled data architect can be very helpful for that purpose. What is a data architect? What is the main difference between a data architect and a data engineer?

Data 87

Data Data Engineering Big Data Architecture 87

CIO

FEBRUARY 14, 2023

An enterprise data ecosystem architected to optimize data flowing in both directions. DataStax Let’s start with the real-time operational data store, as this is the central data engine for building ML models. Learn how DataStax enables real-time AI.

Artificial Inteligence 282

Artificial Inteligence Strategy Data Machine Learning 282

O'Reilly Media - Ideas

AUGUST 18, 2020

That is, products that are laser-focused on one aspect of the data science and machine learning workflows, in contrast to all-in-one platforms that attempt to solve the entire space of data workflows. The Two Cultures of Data Tooling. Lessons Learned from Data Warehouse and Data Engineering Platforms.

Machine Learning 144

Machine Learning Artificial Inteligence Data Data Engineering 144