Expert insights. Personalized for you.
The O’Reilly Data Show Podcast: A special episode to mark the 100th episode. This episode of the Data Show marks our 100th episode. Continue reading The evolution of data science, data engineering, and AI MORE
Cloudera Altus ( launched in May 2017 ) is a platform-as-a-service (PaaS) offering that enables users to analyze and process data at scale in public cloud infrastructures. The post Cloudera Altus on Microsoft Azure appeared first on Cloudera Engineering Blog. MORE
Shared Data Experience — SDX — is Cloudera’s secret ingredient that makes it possible to deploy Cloudera’s four core functions (Data Engineering, Data Science, Analytic DB, Operational DB) on a single platform. What is SDX? MORE
Being at the top of data science capabilities, machine learning and artificial intelligence are buzzing technologies many organizations are eager to adopt. Data science layers towards AI, Source: Monica Rogati. Explaining Data Engineering and Data Warehouse. MORE
Altus empowers customers and partners alike, to run data engineering workloads in the cloud, leveraging cloud infrastructures such as AWS. Cloudera Altus also provides the ability to create data engineering pipelines using both a web console and CLI. MORE
It’s important to understand the differences between a data engineer and a data scientist. Misunderstanding or not knowing these differences are making teams fail or underperform with big data. Overly simplistic venn diagram with data scientists and data engineers. MORE
The O’Reilly Data Show Podcast: Avner Braverman on what’s missing from serverless today and what users should expect in the near future. Continue reading What data scientists and data engineers can do with current generation serverless technologies MORE
Senior Software Engineer – Big Data. IO is the global leader in software-defined data centers. IO technology provides an innovative way to deploy, provision, and optimize data center capacity anywhere in the world based on the needs of businesses, applications, and users. MORE
The former sees growing investment in data analytics to become data-driven (45% of organizations expect to increase their spending in this area) while the latter is fueled by disruptive technology and the adoption of AI (41% of organizations name it as their game changer). MORE
Book Review Artificial Intelligence Machine Learning Programming InfoQ Safety Culture & Methods AI, ML & Data Engineering articleThe book Rebooting AI explains why a different approach other than deep learning is needed to unlock the potential of AI. MORE
I’m interrupting the regular programming for a quick announcement: we’re looking for data engineers at Better. Migrate our data warehouse to Redshift. Write and productionize a web scraper to ingest a bunch of financial third party data. Fit Gamma distributions to conversion data to understand the time lag and conversion rates. This position is very engineering-heavy at its core, and the main qualification is solid programming skills. MORE
Ori Rafael is CEO of Upsolver, which delivers a self-service data lake ETL platform that bridges the gap between data lakes and data consumers and enables organizations to unlock the value of their cloud data lakes. What Does the Data Say? Culture Data Contributed MORE
Is it all about customer-centric business, or is there any data left? Can we integrate data analytics and customer empathy? MORE
A month ago, we publicly announced Cloudera Altus , our new platform–as–a–service offering, and today, we are expanding the Altus data engineering service to support AWS EC2 Spot instances. Cloud Altus aws data engineering ETL Preemptible Spot TCO MORE
The post Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 2 appeared first on Cloudera Engineering Blog. MORE
With the rise of big data and data science, storage and retrieval have become a critical pipeline component for data use and analysis. Recently, new data storage technologies have emerged. Which one is best suited for data engineering? In this article, Toptal Data Scientist Ken Hu compares three prominent storage technologies within the context of data engineering MORE
Or, why science and engineering are still different disciplines. "A He would have to ask an engineer to do it for him.". A few months ago, I wrote about the differences between data engineers and data scientists. Let’s call this data scientist Bob. MORE
Automated testing Artificial Intelligence Programming Automation Development Culture & Methods AI, ML & Data Engineering articleThese days AI is a big buzzword. While it rises in popularity, the controversy surrounding it flourishes as well. MORE
In particular, there has been a significant increase in demand for data scientists. Companies are searching and competing for increasingly scarce data scientists as the […]. Artificial Intelligence Big Data and Analytics Cloud Computing CTO artificial intelligence big data data data engineer data scientist Enterprise MORE
Between growing data and analytics teams, more collaborative projects, and GDPR , security and governance are becoming bigger issues. As organizations' analytical maturity improves, it's increasingly critical that business and technical users alike understand data best practices. MORE
However, as the data warehousing world shifts into a fast-paced, digital, and agile era, the demands to quickly generate reports and help guide data-driven decisions are constantly increasing. New data types need to be quickly joined with existing data sets. MORE
As AI increasingly gains popularity among enterprises, companies are actively seeking data scientists who possess data science skills. Many enterprises confuse the roles of data scientists and data engineers. Artificial Intelligence Big Data and Analytics CTOEven though some traits, skills, programming languages and tools are shared by both roles, the overall roles and core skill sets are different and are not [.]. MORE
A traditional BI and analytics organization consists of three main groups: Analysts that develop reports often using sample data. The data management team – modelers that take requests, find data, and develop models to answer the questions. In a big data world, we often see three new roles emerge and work more closely together: data engineers, data scientists and architects. You can think of them as the data workhorse. MORE
In part 1 of this series we introduced Kentik Data Engine™, the backend to Kentik Detect™, which is a large-scale distributed datastore that is optimized for querying IP flow records (NetFlow v5/9, sFlow, IPFIX) and related network data (GeoIP, BGP, SNMP). MORE
I’m interrupting the regular programming for a quick announcement: we’re looking for data engineers at Better. Migrate our data warehouse to Redshift. Write and productionize a web scraper to ingest a bunch of financial third party data. MORE
Here at Kentik, we’ve applied many of the same concepts to Kentik Data Engine™ (KDE), a datastore optimized for querying IP flow records (NetFlow v5/9, sFlow, IPFIX) and related network data (GeoIP, BGP, SNMP). Next, let’s look at capacity: how big is our “big data”? MORE
Tools like Apache Spark bring scale to machine learning, and Cloudera Data Science Workbench brings Spark to data scientists. What happens when a data scientist wants to burst into the cloud to forge models at scale? MORE
I mentioned in an earlier blog titled, “Staffing your big data team, ” that data engineers are critical to a successful data journey. That said, most companies that are early in their journey lack a dedicated engineering group. Image 1: Data Engineering Skillsets. MORE
David Schaaf explains how data science and data engineering can work together to deliver results to decision makers. Continue reading Jupyter notebooks and the intersection of data science and data engineering MORE
The post Data Engineering: The Heavy Lifting Behind IoT appeared first on QBurst - Blog. This post is part of our continuing blog series on the Internet of Things. In our previous posts, we discussed sensors, wireless technologies in IoT, and Connected Operations: 3 IoT Scenarios. MORE
quintillion bytes of data generated daily, data scientists get busier than ever. And data science provides us with methods to make use of this data. So, along with data scientists who create algorithms, there are data engineers, the architects of data platforms. MORE
Get a basic overview of data engineering and then go deeper with recommended resources. As the the data space has matured, data engineering has emerged as a separate and related role that works in concert with data scientists. MORE
Use Cases Project Management Book Review Artificial Intelligence Machine Learning Programming InfoQ Training / Certification Agile Development Culture & Methods AI, ML & Data Engineering article MORE
It was exactly one year ago at Strata London that we introduced the world to Cloudera Altus Data Engineering. That is what we introduced to AWS users last May – Altus Data Engineering (on AWS). But Altus is more than a multi-cloud Data Engineering cloud service. MORE
With modern businesses dealing with an ever-increasing volume of data, and an expanding set of data sources, the data engineering process that enables analysis, visualization, and reporting only becomes more important. MORE
QCon returned to London this past March for its fourteenth year in the city, attracting over 1,600 senior developers, architects, data engineers, team leads, and CTOs. QCon London 2020 DevOps Development Architecture & Design Culture & Methods AI, ML & Data Engineering article MORE
Programming Languages System Programming Programming DevOps Culture & Methods Development Architecture & Design AI, ML & Data Engineering articleAt the end of 2019, InfoQ ran a survey of our readers to find out what tools, techniques, and languages they were using. MORE
Companies are increasingly moving their data operations into the cloud. With both companies focusing on helping customers derive business insights out of vast amounts of data, our new joint offering will dramatically simplify leveraging cloud-native infrastructures for big data analytics. MORE
Data Engineering Related Topics O'Reilly Media - Data
APRIL 11, 2018
It’s important to understand the differences between a data engineer and a data scientist. Misunderstanding or not knowing these differences are making teams fail or underperform with big data. Overly simplistic venn diagram with data scientists and data engineers.
O'Reilly Media - Data
JULY 16, 2018
Get a basic overview of data engineering and then go deeper with recommended resources. As the the data space has matured, data engineering has emerged as a separate and related role that works in concert with data scientists.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Cloudera Engineering
MAY 24, 2017
With modern businesses dealing with an ever-increasing volume of data, and an expanding set of data sources, the data engineering process that enables analysis, visualization, and reporting only becomes more important.
Erik Bernhardsson
JANUARY 27, 2018
I’m interrupting the regular programming for a quick announcement: we’re looking for data engineers at Better. Migrate our data warehouse to Redshift. Write and productionize a web scraper to ingest a bunch of financial third party data.
O'Reilly Media - Data
MAY 24, 2018
The O’Reilly Data Show Podcast: A special episode to mark the 100th episode. This episode of the Data Show marks our 100th episode. Continue reading The evolution of data science, data engineering, and AI
Erik Bernhardsson
JANUARY 27, 2018
I’m interrupting the regular programming for a quick announcement: we’re looking for data engineers at Better. Migrate our data warehouse to Redshift. Write and productionize a web scraper to ingest a bunch of financial third party data. Fit Gamma distributions to conversion data to understand the time lag and conversion rates. This position is very engineering-heavy at its core, and the main qualification is solid programming skills.
|
Altexsoft
JUNE 25, 2019
Being at the top of data science capabilities, machine learning and artificial intelligence are buzzing technologies many organizations are eager to adopt. Data science layers towards AI, Source: Monica Rogati. Explaining Data Engineering and Data Warehouse.
Cloudera
JANUARY 12, 2018
I mentioned in an earlier blog titled, “Staffing your big data team, ” that data engineers are critical to a successful data journey. That said, most companies that are early in their journey lack a dedicated engineering group. Image 1: Data Engineering Skillsets.
Altexsoft
APRIL 22, 2020
quintillion bytes of data generated daily, data scientists get busier than ever. And data science provides us with methods to make use of this data. So, along with data scientists who create algorithms, there are data engineers, the architects of data platforms.
O'Reilly Media - Data
AUGUST 27, 2018
David Schaaf explains how data science and data engineering can work together to deliver results to decision makers. Continue reading Jupyter notebooks and the intersection of data science and data engineering
The New Stack
FEBRUARY 20, 2020
Ori Rafael is CEO of Upsolver, which delivers a self-service data lake ETL platform that bridges the gap between data lakes and data consumers and enables organizations to unlock the value of their cloud data lakes. What Does the Data Say? Culture Data Contributed
|
O'Reilly Media - Ideas
APRIL 11, 2019
The O’Reilly Data Show Podcast: Avner Braverman on what’s missing from serverless today and what users should expect in the near future. Continue reading What data scientists and data engineers can do with current generation serverless technologies
CTOvision
APRIL 27, 2018
In particular, there has been a significant increase in demand for data scientists. Companies are searching and competing for increasingly scarce data scientists as the […]. Artificial Intelligence Big Data and Analytics Cloud Computing CTO artificial intelligence big data data data engineer data scientist Enterprise
QBurst
FEBRUARY 12, 2019
The post Data Engineering: The Heavy Lifting Behind IoT appeared first on QBurst - Blog. This post is part of our continuing blog series on the Internet of Things. In our previous posts, we discussed sensors, wireless technologies in IoT, and Connected Operations: 3 IoT Scenarios.
CTOvision
MAY 1, 2014
Senior Software Engineer – Big Data. IO is the global leader in software-defined data centers. IO technology provides an innovative way to deploy, provision, and optimize data center capacity anywhere in the world based on the needs of businesses, applications, and users.
Toptal
DECEMBER 6, 2016
With the rise of big data and data science, storage and retrieval have become a critical pipeline component for data use and analysis. Recently, new data storage technologies have emerged. Which one is best suited for data engineering? In this article, Toptal Data Scientist Ken Hu compares three prominent storage technologies within the context of data engineering
Kentik
MAY 2, 2016
In part 1 of this series we introduced Kentik Data Engine™, the backend to Kentik Detect™, which is a large-scale distributed datastore that is optimized for querying IP flow records (NetFlow v5/9, sFlow, IPFIX) and related network data (GeoIP, BGP, SNMP).
Kentik
APRIL 25, 2016
Here at Kentik, we’ve applied many of the same concepts to Kentik Data Engine™ (KDE), a datastore optimized for querying IP flow records (NetFlow v5/9, sFlow, IPFIX) and related network data (GeoIP, BGP, SNMP). Next, let’s look at capacity: how big is our “big data”?
InfoQ Culture Methods
MAY 4, 2020
Automated testing Artificial Intelligence Programming Automation Development Culture & Methods AI, ML & Data Engineering articleThese days AI is a big buzzword. While it rises in popularity, the controversy surrounding it flourishes as well.
CTOvision
JUNE 7, 2018
As AI increasingly gains popularity among enterprises, companies are actively seeking data scientists who possess data science skills. Many enterprises confuse the roles of data scientists and data engineers. Artificial Intelligence Big Data and Analytics CTOEven though some traits, skills, programming languages and tools are shared by both roles, the overall roles and core skill sets are different and are not [.].
InfoQ Culture Methods
APRIL 7, 2020
QCon returned to London this past March for its fourteenth year in the city, attracting over 1,600 senior developers, architects, data engineers, team leads, and CTOs. QCon London 2020 DevOps Development Architecture & Design Culture & Methods AI, ML & Data Engineering article
InfoQ Culture Methods
MARCH 19, 2020
Use Cases Project Management Book Review Artificial Intelligence Machine Learning Programming InfoQ Training / Certification Agile Development Culture & Methods AI, ML & Data Engineering article
Cloudera Engineering
JUNE 23, 2017
A month ago, we publicly announced Cloudera Altus , our new platform–as–a–service offering, and today, we are expanding the Altus data engineering service to support AWS EC2 Spot instances. Cloud Altus aws data engineering ETL Preemptible Spot TCO
InfoQ Culture Methods
APRIL 18, 2020
Programming Languages System Programming Programming DevOps Culture & Methods Development Architecture & Design AI, ML & Data Engineering articleAt the end of 2019, InfoQ ran a survey of our readers to find out what tools, techniques, and languages they were using.
Cloudera Engineering
FEBRUARY 22, 2018
Altus empowers customers and partners alike, to run data engineering workloads in the cloud, leveraging cloud infrastructures such as AWS. Cloudera Altus also provides the ability to create data engineering pipelines using both a web console and CLI.
InfoQ Culture Methods
SEPTEMBER 6, 2019
Is it all about customer-centric business, or is there any data left? Can we integrate data analytics and customer empathy?
Dataiku
AUGUST 9, 2019
Between growing data and analytics teams, more collaborative projects, and GDPR , security and governance are becoming bigger issues. As organizations' analytical maturity improves, it's increasingly critical that business and technical users alike understand data best practices.
Cloudera
FEBRUARY 11, 2019
The former sees growing investment in data analytics to become data-driven (45% of organizations expect to increase their spending in this area) while the latter is fueled by disruptive technology and the adoption of AI (41% of organizations name it as their game changer).
Cloudera Engineering
SEPTEMBER 26, 2017
Shared Data Experience — SDX — is Cloudera’s secret ingredient that makes it possible to deploy Cloudera’s four core functions (Data Engineering, Data Science, Analytic DB, Operational DB) on a single platform. What is SDX?
InfoQ Culture Methods
NOVEMBER 21, 2019
Book Review Artificial Intelligence Machine Learning Programming InfoQ Safety Culture & Methods AI, ML & Data Engineering articleThe book Rebooting AI explains why a different approach other than deep learning is needed to unlock the potential of AI.
Cloudera Engineering
SEPTEMBER 27, 2017
Cloudera Altus ( launched in May 2017 ) is a platform-as-a-service (PaaS) offering that enables users to analyze and process data at scale in public cloud infrastructures. The post Cloudera Altus on Microsoft Azure appeared first on Cloudera Engineering Blog.
Cloudera Engineering
OCTOBER 6, 2017
Companies are increasingly moving their data operations into the cloud. With both companies focusing on helping customers derive business insights out of vast amounts of data, our new joint offering will dramatically simplify leveraging cloud-native infrastructures for big data analytics.
Cloudera Engineering
NOVEMBER 15, 2017
Tools like Apache Spark bring scale to machine learning, and Cloudera Data Science Workbench brings Spark to data scientists. What happens when a data scientist wants to burst into the cloud to forge models at scale?
Cloudera
JANUARY 3, 2018
A traditional BI and analytics organization consists of three main groups: Analysts that develop reports often using sample data. The data management team – modelers that take requests, find data, and develop models to answer the questions. In a big data world, we often see three new roles emerge and work more closely together: data engineers, data scientists and architects. You can think of them as the data workhorse.
Cloudera
MAY 22, 2018
It was exactly one year ago at Strata London that we introduced the world to Cloudera Altus Data Engineering. That is what we introduced to AWS users last May – Altus Data Engineering (on AWS). But Altus is more than a multi-cloud Data Engineering cloud service.
Cloudera
SEPTEMBER 17, 2018
However, as the data warehousing world shifts into a fast-paced, digital, and agile era, the demands to quickly generate reports and help guide data-driven decisions are constantly increasing. New data types need to be quickly joined with existing data sets.
Cloudera Engineering
JANUARY 23, 2018
The post Deploy Cloudera EDH Clusters Like a Boss Revamped – Part 2 appeared first on Cloudera Engineering Blog.
169 
Let's personalize your content