Many architectures contain a division where data analytics / data warehouse systems are separated from the applications that they analyze. This requires transfers of data between these systems at significant cost. Is that separation worth it?

To begin to answer this we might consider why organizations create applications. Simply put, an organization has some goal -- some way in which it would like to change the world -- and it uses applications to achieve that goal. Now we might briefly generalize this a bit further, and ask -- how do organizations change the world in general?

Organizations make decisions, and then they take actions based on those decisions. Their actions may vary in character but ultimately an organization acts by spending money. To decide how to invest, organizations need to collect data, and the immediate goal of their investments is to change that data. This is a process of experimentation that is intended to steer some part of the world in a positive direction.

It follows that the only purpose of an application is to collect data. All the user interface elements, etc. are there merely as part of an experiment intended to move the data in one direction or another. Applications are data ingestion devices.

By these lights it is perhaps worth fuzzing the boundary between application development and data engineering/analytics. There may be significant economies to be found by reducing the impedance between these systems. With regard to software development frameworks, the incorporation of data analytics concerns into the core of software frameworks could have many benefits.