Recently I was requested by a client to help with a legacy data migration to Oracle. The data resided in flat files generated from an IBM AS400 in csv and other text formats. Part of the migration was to read through the text lines that contained lines of text which needed to be tokenized and then inserted into different columns. What follows are the atomic PL/SQL functions that I wrote to break down the legacy data lines into tokens which can then be manipulated and used to populate the normalized target schema. I hope you find this useful as a foundation to develop more complex parsing algorithms.