19 answers
1
Question
Asked
5626 views
What is the difference between the job of a Data Engineer and a Data analytics ?
I am currently applying to internships, and there are two roles available. These roles seem similar, so I am wondering if there is anyone from the industry that can further explain. Thank you. #GivingisCaring #dataengineering #engineering #data
Login to comment
19 answers
Updated
Angela’s Answer
Hi Yazmin P. - great question! I work very closely with both two teams and hope my answer can help a little bit here! =)
1. Data Analytics: transform data into consumable insights
- Data Analysis: i.e., use data to help answer business questions such as why we lost 10% more customers in June vs. May?
- Metrics Reporting: i.e., how many new customers did we acquire last month? How did that compare to last year same period?
- Data visualization / dashboard: i.e., build dashboard to show trend over certain time periods; compare performance of multiple products etc.
2. Data Engineer: build and maintain scalable data infrastructure; take raw data into structured format/model
- Data Integration: companies normally uses multiple systems/softwares and we need to connect all the data together. From Salesforce, to Stripe to Intercom etc. The Data Engineer need to ensure we collect all the data we needed
- Data warehouse / Data pipeline management : ensure data warehouse is reliable and performant
- Data models/schema development: build out data infrastructure that is scalable and easy to use by the Analytics team or other parts of the organization
1. Data Analytics: transform data into consumable insights
- Data Analysis: i.e., use data to help answer business questions such as why we lost 10% more customers in June vs. May?
- Metrics Reporting: i.e., how many new customers did we acquire last month? How did that compare to last year same period?
- Data visualization / dashboard: i.e., build dashboard to show trend over certain time periods; compare performance of multiple products etc.
2. Data Engineer: build and maintain scalable data infrastructure; take raw data into structured format/model
- Data Integration: companies normally uses multiple systems/softwares and we need to connect all the data together. From Salesforce, to Stripe to Intercom etc. The Data Engineer need to ensure we collect all the data we needed
- Data warehouse / Data pipeline management : ensure data warehouse is reliable and performant
- Data models/schema development: build out data infrastructure that is scalable and easy to use by the Analytics team or other parts of the organization
Updated
Mitchell’s Answer
The names can vary --- there's still conflict over what's statistics and what's data science! - but the thing to remember is that there are tasks in the pipeline.
Think of it as buying a cake. Someone has to harvest the wheat, grind it to flour, ship the flour, and finally mix and bake the cake.
Data engineering could be seen as getting the ingredients together and checking for quality -- is everything in the data that's supposed to be, did anything get in that wasn't supposed to? Then the analytics is taking the otherwise inedible raw ingredients (who snacks on flour?) and turning them into something intellectually digestible.
What to consider at this point in your career is what tools and problems get you interested. Do you like the technical details of a database? The creative aspects of data visualization?
Also none of these roles are in a vacuum. The engineer and the analytics person will be working together on many projects.
Think of it as buying a cake. Someone has to harvest the wheat, grind it to flour, ship the flour, and finally mix and bake the cake.
Data engineering could be seen as getting the ingredients together and checking for quality -- is everything in the data that's supposed to be, did anything get in that wasn't supposed to? Then the analytics is taking the otherwise inedible raw ingredients (who snacks on flour?) and turning them into something intellectually digestible.
What to consider at this point in your career is what tools and problems get you interested. Do you like the technical details of a database? The creative aspects of data visualization?
Also none of these roles are in a vacuum. The engineer and the analytics person will be working together on many projects.
Updated
Nicole’s Answer
Hi Yazmin P. Thanks so much for your question.
I think one of the differences (and one of the cooler things) about a data engineer vs data analytics is that data analytics allows you to learn and use some pretty interesting visualization tools. In learning these tools (like Tableau, R, Python) you begin to make a shift from the area of engineering, as a set of specific steps, to analytics where you can see and show what is happening over time. That "what is happening" can yield important aha moments whether it is to determine how often something is happening, to whom something is happening and whether or not what is happening is a good thing.
The concept of a data engineer is a little new to me but if it is what I suspect, it will likely be a great foundation for understanding how to build a sustainable process that ensures good data to be used to support great analytics.
Hope you find this answer helpful. Best of luck to you!
I think one of the differences (and one of the cooler things) about a data engineer vs data analytics is that data analytics allows you to learn and use some pretty interesting visualization tools. In learning these tools (like Tableau, R, Python) you begin to make a shift from the area of engineering, as a set of specific steps, to analytics where you can see and show what is happening over time. That "what is happening" can yield important aha moments whether it is to determine how often something is happening, to whom something is happening and whether or not what is happening is a good thing.
The concept of a data engineer is a little new to me but if it is what I suspect, it will likely be a great foundation for understanding how to build a sustainable process that ensures good data to be used to support great analytics.
Hope you find this answer helpful. Best of luck to you!
Updated
Sikandar’s Answer
Data Engineers and Data Analysts are 2 different careers although they primarily deal with data.
Data Engineers can work with tools like Azure Data Engineer, Big Data Engineer or ETL tools like Informatica, Abinitio.
Data Engineers in most of the jobs doesn't need to know programming but would be an asset if they know programming languages like python if they want to branch to Big Data Engineering or Azure Data Engineering with scala.
One could start by learning SQL and any ETL tool or Cloud ETL tool . If some like programming, one can learn python that can be used with Big Data Engineer/Azure Data Engineer learning and also useful on spark.
They primarily work with data and do ETL(Extract data from sources, transform the data and the load the data to the targets).
Data Engineers doesn't need to know the BI tools like Tableau as that's a complete different career of BI Developers.
Data Analysts use the available data to generate reports using tools like tableau/excel and also could be using SQL with DB. They could be using python but some roles doesn't need.
Data Engineers can work with tools like Azure Data Engineer, Big Data Engineer or ETL tools like Informatica, Abinitio.
Data Engineers in most of the jobs doesn't need to know programming but would be an asset if they know programming languages like python if they want to branch to Big Data Engineering or Azure Data Engineering with scala.
One could start by learning SQL and any ETL tool or Cloud ETL tool . If some like programming, one can learn python that can be used with Big Data Engineer/Azure Data Engineer learning and also useful on spark.
They primarily work with data and do ETL(Extract data from sources, transform the data and the load the data to the targets).
Data Engineers doesn't need to know the BI tools like Tableau as that's a complete different career of BI Developers.
Data Analysts use the available data to generate reports using tools like tableau/excel and also could be using SQL with DB. They could be using python but some roles doesn't need.
Updated
Jessica’s Answer
Data Analytics is about understanding the meaning of the data, making sense of it, and finding trends. This involves pulling data from the database via SQL, working with spreadsheet, use R or Python to script some of the data prepping for visualization, which can be piped to tableau or power BI to create and share useful dashboards.
Data Engineer is more about data ETL (extract, transform, load) basically this means taking the data and moving it between systems or databases, or from spreadsheet / feeds to structured forms (like relational database) At this point the data engineer may be given instructions to apply additional transformation on the data so that it is functional for the end users (data analyst / researcher)
In most cases, the data engineer and data analyst will work together very closely. Data Engineer to source and to structure the data and make it available for the data analysts to consume the data and to make sense of everything via queries, scripts, and visualization tools.
Data Engineer is more about data ETL (extract, transform, load) basically this means taking the data and moving it between systems or databases, or from spreadsheet / feeds to structured forms (like relational database) At this point the data engineer may be given instructions to apply additional transformation on the data so that it is functional for the end users (data analyst / researcher)
In most cases, the data engineer and data analyst will work together very closely. Data Engineer to source and to structure the data and make it available for the data analysts to consume the data and to make sense of everything via queries, scripts, and visualization tools.
Updated
Nick’s Answer
Hi Yazmin, great question! Data engineers can be thought of as architects in the sense that they are concerned with the designing and building of features that work to prepare data that can be then used by data analysts/data scientists who work to derive meaning from the data passing through these feature and predict future outcomes based on the data they are interpreting.
Updated
Henry’s Answer
Data engineering is mostly about data capture/storage/movement and building efficient software systems which do the previous 3 things. Data analytics is about taking that data which a data engineer built and turning it into useful business insights with respect to their line of business. The two ideally should interact with each other in order to intelligently guide the overall data process.
Updated
GIRIDHAR RAO’s Answer
Yazmin,
Data Engineering will coverts/transforms unstructured & raw data into a structured and more consumable format for data analytics.
Data Analytics provides insights into the data that help businesses in making better, more accurate and informed decisions
Data Engineering will coverts/transforms unstructured & raw data into a structured and more consumable format for data analytics.
Data Analytics provides insights into the data that help businesses in making better, more accurate and informed decisions
Updated
Ali’s Answer
Hi Yazmin,
Great question!
First off as you apply to internships, regardless of the job title, I would recommend you read the job description, as that will give you the most accurate representation of what you'll do in that role. Sometimes companies can use similar titles but the actual day-to-day could look different. Also, depending on the size of the company and the maturity of the data organizations, it could very well be that someone plays a little bit of both roles (e.g. a start up or a new data department).
That said, at a high level, in my experience the Data Engineers do amazing work to build data pipelines, automate jobs, create datasets, etc. Data Analysts & Data Scientists can then take these data assets to perform analysis.
Hope that helps!
Great question!
First off as you apply to internships, regardless of the job title, I would recommend you read the job description, as that will give you the most accurate representation of what you'll do in that role. Sometimes companies can use similar titles but the actual day-to-day could look different. Also, depending on the size of the company and the maturity of the data organizations, it could very well be that someone plays a little bit of both roles (e.g. a start up or a new data department).
That said, at a high level, in my experience the Data Engineers do amazing work to build data pipelines, automate jobs, create datasets, etc. Data Analysts & Data Scientists can then take these data assets to perform analysis.
Hope that helps!
Updated
Gururaj’s Answer
It is good question. If you are really passion about programming and transformational roles that can bring more value to organization and your career, you could choose to be Data analysts. Today's information technology is looking for more machine learning, deep learning skill sets which are more prominent skills required for Data analysts. The data analysts or data scientists are really back bone of some of the key transformation happening in the social media and retail industry business. Data analysts are in demand because of the unique skills that they bring to organization where they analyse the trend and build the predictable models to transform the business. So, if you are willing to learn those technology (AI/ML), then I would suggest you to be Data analysts.
Updated
JinYeop’s Answer
Dear Yazmin,
I think you are having a general question in data working area.
As you may know, there are a few jobs in data field, data analyst, data scientist, data engineer, data steward etc.
Among these, data analyst is working to solve "what & why happened" by SQL or Visualization in a certain business.
On the other hand, data engineer is working to optimize and organize "Data" for data scientist and data analyst.
IT / DB system management is a kind of main role of data engineer. Descriptive analytics and Diagnostic analytics are the key role of data analyst.
Even though both data jobs have a bit different role, but both jobs should be closely working together to come up with data solutions and works.
I think you are having a general question in data working area.
As you may know, there are a few jobs in data field, data analyst, data scientist, data engineer, data steward etc.
Among these, data analyst is working to solve "what & why happened" by SQL or Visualization in a certain business.
On the other hand, data engineer is working to optimize and organize "Data" for data scientist and data analyst.
IT / DB system management is a kind of main role of data engineer. Descriptive analytics and Diagnostic analytics are the key role of data analyst.
Even though both data jobs have a bit different role, but both jobs should be closely working together to come up with data solutions and works.
Updated
Jaskarn’s Answer
Data Analysts tell stories from the data they receive from Data Engineers. Data Engineers deal with the back-end where they extract, transform and load the data (ETL process), while a Data Analyst would tell the story from the data they receive.
Some technical skills and languages to have in your back-pack:
-SQL, NoSQL
-Python
BI Tools for a Data Analyst in the Industry:
-Tableau, Looker, Periscope
Some free website to get you started: -https://www.freecodecamp.org/ -https://www.datacamp.com/ -https://www.codewars.com/
Some technical skills and languages to have in your back-pack:
-SQL, NoSQL
-Python
BI Tools for a Data Analyst in the Industry:
-Tableau, Looker, Periscope
Jaskarn recommends the following next steps:
Updated
Joseph’s Answer
Data analysts perform analysis. Data engineers create the methods to collect and analysis methods.
Updated
Jaskarn’s Answer
Data Analysts tell stories from the data they receive from Data Engineers. Data Engineers deal with the back-end where they extract, transform and load the data (ETL process). It is great to know both, as these are usually intertwined. You could add Data Science as well to this question. All great paths related to Data.
Some technical skills and languages to have in your back-pack:
-SQL, NoSQL
-Python
BI Tools for a Data Analyst :
-Tableau, Looker, Periscope
Once you learn one language or tool, they are pretty interchangeable with syntax differences.
Some free website to get you started: -https://www.freecodecamp.org/ -https://www.datacamp.com/ -https://www.codewars.com/
Some technical skills and languages to have in your back-pack:
-SQL, NoSQL
-Python
BI Tools for a Data Analyst :
-Tableau, Looker, Periscope
Once you learn one language or tool, they are pretty interchangeable with syntax differences.
Jaskarn recommends the following next steps:
Updated
Paul Anthony’s Answer
These are two roles that generally work together. Most good engineers are serviceable analysts, and visa versa so knowing how to work with the other role is incredibly helpful. I would suggest that if you like things to be neat and organized, if you like having an more immediate, black and white feedback, then engineering is probably better for you. If you can be comfortable with nebulous instructions, can sell your ideas, and don't need immediate right/wrong feedback, than you make for a good analyst.
Updated
Joe’s Answer
Like the above answers, ususally Data Analysts consume the data and make recommendations based on what they see. Data Engineers ensure the data that Analysts use is formatted and available for users. One thing to think of as well is that Data Analysts typically are more people facing (internal / customers) than Data Engineering. If you're comfortable speaking in front of people about your analysis then Data Analysis is a fun role.
Updated
Yiyi’s Answer
Hi Yazmin,
Great question.
I was considering the questions just two years ago when I graduated from graduate school with a master degree in Business Analytics. I studied Data Analytics and some Data Engineer there. I got a few job offers - Data Analyst/Data Scientist and Data Engineer. In the end, I chose my current role Data Analyst (Business Optimization Analyst). Why?
I realized I enjoyed:
- working on the Business side
- want to be the bridge that connects data and business
- in a client-facing environment
- love to see my work has a direct impact
- can shape the business strategy by showing data-driven insights
- love to present to people
- love to solve a business question using data as a tool
If you are like me, maybe data analytics would be a good option to try it out. If you are also interested in any other side, you can always reach out to people here/on LinkedIn/at your job to understand more about other's job and responsibility.
Hope this helps!
Yiyi
Great question.
I was considering the questions just two years ago when I graduated from graduate school with a master degree in Business Analytics. I studied Data Analytics and some Data Engineer there. I got a few job offers - Data Analyst/Data Scientist and Data Engineer. In the end, I chose my current role Data Analyst (Business Optimization Analyst). Why?
I realized I enjoyed:
- working on the Business side
- want to be the bridge that connects data and business
- in a client-facing environment
- love to see my work has a direct impact
- can shape the business strategy by showing data-driven insights
- love to present to people
- love to solve a business question using data as a tool
If you are like me, maybe data analytics would be a good option to try it out. If you are also interested in any other side, you can always reach out to people here/on LinkedIn/at your job to understand more about other's job and responsibility.
Hope this helps!
Yiyi
Updated
Mehmet’s Answer
There are many roles in the Data Profession, and you can make choices (and evolve your career over time) based on what you want to do and with what you want to work.
You can work with Data or work with Technology.
- Data Analysts work with Data. You can use query, visualization, profiling, statistical tools to look for and find patterns in data and communicate it's meaning, typically focused on one or more 'business needs"
- Data Engineers work with Technology. You would use code in order find, integrate, transform data so it is meaningful
While there is a bit of task overlap between the above two, where you spend 80% of your time varies.
For your reference, there are other Data Professional roles as well.
If you like precision and want to make sure specifications are defined or adhered to, Data Governance or Compliance Manager, or Auditor
If you don't mind repeated tasks with variable information to increase depth of a domain: Data Steward to ensure quality and compliance with specs
Data Architect, Integration Architect, Database Administrator are also related roles.
You can work with Data or work with Technology.
- Data Analysts work with Data. You can use query, visualization, profiling, statistical tools to look for and find patterns in data and communicate it's meaning, typically focused on one or more 'business needs"
- Data Engineers work with Technology. You would use code in order find, integrate, transform data so it is meaningful
While there is a bit of task overlap between the above two, where you spend 80% of your time varies.
For your reference, there are other Data Professional roles as well.
If you like precision and want to make sure specifications are defined or adhered to, Data Governance or Compliance Manager, or Auditor
If you don't mind repeated tasks with variable information to increase depth of a domain: Data Steward to ensure quality and compliance with specs
Data Architect, Integration Architect, Database Administrator are also related roles.
Updated
Akshun’s Answer
Hello there!
The interpretation of roles can differ from one organization to another, or even from one role to another. This is because there are many overlapping characteristics between the two roles, especially since they both revolve around data and require a similar set of skills and understanding of data.
Imagine a Venn diagram representing these two roles.
The overlapping section would include:
- Proficiency in SQL, Python, and similar tools.
- A solid grasp of data and how to manipulate it.
Specific to a Data Analyst:
- They extract answers from the data.
- They need a deeper understanding of the business domain.
- They might be adept at statistical analysis and similar tools.
Specific to a Data Engineer:
- Their main focus is on how data is stored and managed.
- Their skills might encompass data warehousing, data modeling, ingestion, and egress.
- They need a strong understanding of the platforms that generate, move, and store data.
I hope this clarifies things for you.
The interpretation of roles can differ from one organization to another, or even from one role to another. This is because there are many overlapping characteristics between the two roles, especially since they both revolve around data and require a similar set of skills and understanding of data.
Imagine a Venn diagram representing these two roles.
The overlapping section would include:
- Proficiency in SQL, Python, and similar tools.
- A solid grasp of data and how to manipulate it.
Specific to a Data Analyst:
- They extract answers from the data.
- They need a deeper understanding of the business domain.
- They might be adept at statistical analysis and similar tools.
Specific to a Data Engineer:
- Their main focus is on how data is stored and managed.
- Their skills might encompass data warehousing, data modeling, ingestion, and egress.
- They need a strong understanding of the platforms that generate, move, and store data.
I hope this clarifies things for you.