Checking for CSV Column Headers in C#
Importing CSV files into a .NET object isn’t too tricky and for this I usually use CsvHelper. However, sometimes I need to import a CSV file and only extract a couple of columns from it and these columns aren’t always guaranteed to exist. In order to do this I use the following code.
string filePath = "C:/Temp/test.csv"
// Check if file exists.
if (File.Exists(filePath))
{
// Read column headers from file
CsvReader csv = new CsvReader(File.OpenText(filePath));
csv.Read();
csv.ReadHeader();
List<string> headers = csv.Context.HeaderRecord.ToList();
// Read csv into datatable
System.Data.DataTable dataTable = new System.Data.DataTable();
foreach (string header in headers)
{
dataTable.Columns.Add(new System.Data.DataColumn(header));
}
// Check all required columns are present
if (!headers.Exists(x => x == "ua_device_type"))
{
throw new ArgumentException("ua_device_type field not present in input file.");
}
else if (!headers.Exists(x => x == "ua_channel"))
{
throw new ArgumentException("ua_channel field not present in input file.");
}
else if (!headers.Exists(x => x == "message_type"))
{
throw new ArgumentException("message_type field not present in input file.");
}
// Import csv
while (csv.Read())
{
System.Data.DataRow row = dataTable.NewRow();
foreach (System.Data.DataColumn column in dataTable.Columns)
{
row[column.ColumnName] = csv.GetField(column.DataType, column.ColumnName);
}
dataTable.Rows.Add(row);
}
Log.Logger.Debug("File loaded into datatable");
// Make sure all required columns are populated
if (dataTable.Select("ua_channel = '' OR ua_channel = null").Count() > 0)
{
throw new ArgumentException("Output contains null ua_channel values.");
}
if (dataTable.Select("ua_device_type = '' OR ua_device_type = null").Count() > 0)
{
throw new ArgumentException("Output contains null ua_device_type values.");
}
if (dataTable.Select("message_type = '' OR message_type = null OR message_type not in ('push', 'message centre')").Count() > 0)
{
throw new ArgumentException("Output contains null or invalid message_type values.");
}
}
7 Comments
Areeg · 20th September 2018 at 1:41 pm
Please, could you explain what is Log.Logger.Debug(“File loaded into datatable”); engageFile.Rows = dataTable.Rows.Count;
Thanks
Shinigami · 20th September 2018 at 4:15 pm
Hi Areeg, these are just some internal logging messages, they’re not needed for the loading of the files so can be left out of your code.
Areeg · 26th September 2018 at 7:02 am
Thank you for your reply
RamachandraReddy · 26th September 2018 at 9:39 pm
Hi I was trying to do the same thing, But I am getting an error near to the csvreader (Can you please provide the class definition of the csvreader)
Shinigami · 27th September 2018 at 9:50 am
Hi, the CsvReader comes from the NuGet package linked at the top of the post.
JustinUzzanti · 8th July 2019 at 4:45 pm
I’m getting an error for
engageFile.rows = dataTable.Rows.Count;
engageFile is throwing the flag and reads doesnt exist
Shinigami · 8th July 2019 at 5:27 pm
engageFile was an internal object I was using in this project, the code above is just a snippet so doesn’t include the code for creating it but it can be safety ignored. I’ll remove it from the post to avoid further confusion.