tl;dr for various ressource I read, heard or saw
We fucked up big time using Hadoop and other solution without knowing them or thinking about the long term.
Going forward when you read “update” think “delete and replace”, because once transformed and loaded data should be seen as immutable.
To convert data you need to understand
Specific need of the requester
The platform of a company is the result of multiple factors (existing team, legacy and budget) so even if you are in the same context it didn’t mean you should aim for the same solution.
How the data will be vizualised is part of a data engineer job.
Feel in sync with his definition of a DE and I recommend to read the articles he mentions.
Especially the one from Maxime Beauchemin great inspiration as always.