Data defragmentation
CIMS data was stored in multiple file sources, including flat text files, mapping text files, b-tree file structures, and oracle database data extracts with over 1000 tables.
Data volume
Billions of records stored over 20 years for over 50,000 patients needed to be reindexed in a new system with novel requirements such as sub-second searching, data analytics, reporting and metrics.
Mappings
Manual cross mapping between text files, b-tree records, and database extract:

1000’s of text files

50,000+ Patients

400,000,000+ Patients raw data

2,000,000,000+ eletrônic chart inputs
Infostructure
Only single VM was available to process and import all data. TC ES exports had to fine tune the ES to make sure the system would import data without running out of memory which was a challenge to accomplish.