Reddit mentions of The Data Warehouse Lifecycle Toolkit

First of all, I don't know how large your company is or how much data exactly you are dealing with, but I'll give some general advice based on what you've said.

It sounds like your company operates the way many do: with individual data marts that were created organically for different needs as they arose. What you need is a centralized data warehouse that brings these data marts together into some type of star-schema setup. Pick up this book. It's basically an introduction to dimensional modeling but more importantly it lays out how to navigate the politics of a large organization such as yours to get a data warehouse created. You will need someone at C-level most likely to make the push but the benefits are well worth it. It's not a small project, you'll need people to admit you need a major overhaul and be willing to invest in it.

>but getting these people to allow me to connect to the data is the most difficult thing (why is that?!).

Probably because whoever is managing the data realizes that there is no way an end user can make useful queries into the database due to it's disjointed and poorly maintained manner.

>In fact, I think they just hired some external enterprise data company, which I feel like is the wrong approach. Information is the most important thing. I feel like this is one thing a company should be managing in-house and not outsourcing. Is that completely wrong?

I would tend to agree, it's important to have people in the organization that understand the data structure fully. It's not unusual to bring in consultants to provide expertise on specific things (like the ETL process) but if I were a large financial company I would want our in house team to have a good handle on the data coming in and being stored.

>Fields are constantly being overwritten so that history isn't maintained. People aren't notified of the overwritten name changes so existing reports aren't capturing all information properly.

At the very least convince them to create expiration fields so you can expire old records without losing information.