Simple CSV to DataGrid Command

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

mrcoollion
Posts: 738
Joined: Thu Sep 11, 2014 1:49 pm

Simple CSV to DataGrid Command

Post by mrcoollion » Wed Mar 12, 2025 6:32 pm

Hello LC friends,

For those who are interested and want to have a simple way to add CSV file data to a datagrid you can use this code.
I was in need of a very simple way to create a datagrid and add, change and remove data So i made the below routine for this.
I can probably be improved a lot but it does work.

CSV Data must be comma and LF delimited. But this is easy to change for other CSV file types with the replace function of LiveCode.

For testing:
1) Copy the code into the script of a button.
2) Put a datagrid in your interface and give it a name.
3) Then replace <YourDataGridName> with the name of your DataGrid.

Test the different options and please forgive me any mistakes ;-)

Regards,

Paul (MrCoolLion)

Code: Select all

on mouseUp pMouseButton
   put "ID,Colom A,Colom B,Colom C"&lf&"A01,Info A,Info B,Info C"&lf&"A02,Info D,Info E,Info F" into tFileData
   //put "ID,Colom A,Colom B,Colom C"&lf&"A03,Info J,Info K,Info L" into tAddLineData
   //put "ID,Colom A,Colom B,Colom C"&lf&"A02,Info UJ,Info UK,Info UL" into tUpdateLineData // Whole line example first item must be Unique identifier.
   //put "ID,Colom B"&lf&"A02,Info UPD" into tUpdateLineData // Change only one item in a line first item must be Unique identifier.
   //put "ID"&lf&"A02" into tDeleteLineData // first item must be Unique identifier.
   -----------------------------------------------------------------------------
   //put "Clear" into tTask
   put "Create" into tTask // tFileData  first item must be Unique identifier.//put "ID,Colom A,Colom B,Colom C"&lf&"A01,Info A,Info B,Info C"&lf&"A02,Info D,Info E,Info F" into tFileData
   --
   //put "AddLine" into tTask //put "ID,Colom A,Colom B,Colom C"&lf&"A03,Info J,Info K,Info L" into tAddLineData
   //put tAddLineData into tFileData
   --
   //put "UpdateLine" into tTask //put "ID,Colom B"&lf&"A02,Info UPD" into tUpdateLineData // first item must be Unique identifier.
   //put tUpdateLineData into tFileData
   --
   //put "DeleteLine" into tTask //put "ID"&lf&"A02" into tDeleteLineData // first item must be Unique identifier.
   //put tDeleteLineData into tFileData
   --
   put "<YourDataGridName>" into tDataGridName
   DataGridCreateColumnsFillFromCSV tDataGridName, tFileData, tTask // "Clear" or "Create" or AddLine or UpdateLine or DeleteLine ; If Task is Create or Update Firstline must be column Names also file must be comma and lf seperated, 
   --
end mouseUp

command DataGridCreateColumnsFillFromCSV tDataGridName, tFileData, tTask 
   // With this routine you can fill an empty datagrid from a csv file, add a line, update a line or line items, and delete a line 
   // "Clear" or "Create" or "AddLine" or "UpdateLine" or DeleteLine 
   // If Task is "Create"  Firstline must be column Names also file must be comma seperated and lf delimited, 
   // if Task is "Add" or "Update" Firstline must be column Names then lf then Columns Data
   // if Task is "DeleteLine" Firstline must be column ID Name then lf then Line ID Data
   -- Examples --
   //put "ID,Colom A,Colom B,Colom C"&lf&"A01,Info A,Info B,Info C"&lf&"A02,Info D,Info E,Info F" into tFileData
   //put "ID,Colom A,Colom B,Colom C"&lf&"A03,Info J,Info K,Info L" into tAddLineData
   //put "ID,Colom A,Colom B,Colom C"&lf&"A02,Info UJ,Info UK,Info UL" into tUpdateLineData // Only per line not idividual item first item must be Unique identifier.
   //put "ID,Colom B"&lf&"A02,Info UPD" into tUpdateLineData // first item must be Unique identifier.
   //put "ID"&lf&"A02" into tDeleteLineData // first item must be Unique identifier.
   --
   //put "Clear" into tTask
   //put "Create" into tTask // tFileData
   --
   //put "AddLine" into tTask
   //put tAddLineData into tFileData
   --
   //put "UpdateLine" into tTask
   //put tUpdateLineData into tFileData
   --
   //put "DeleteLine" into tTask
   //put tDeleteLineData into tFileData
   ----------------------------------------------------
   if tTask = "Clear" or tTask = "Create"
   then
      set the dgProps["columns"] of control tDataGridName to "" // Empty DataGrid Columns
      set the dgData of group tDataGridName to "" // Empty DataGrid Data
      if tTask = "Clear"
      then
         exit DataGridCreateColumnsFillFromCSV
      end if
   end if
   --------------------------------------------------
   if tTask = "AddLine"
   then
      put the dgNumberOfLines of group tDataGridName into tLineNbrs
      if tLineNbrs = 0 
      then 
         put "Create" into tTask
      end if
   end if
   ----------------------------------------------------
   if  tTask = "Create"
   then
      put line 1 of tFileData into tLine1Data
      put tLine1Data into tLine1CRData
      replace "," with cr in tLine1CRData
      set the dgProps["columns"] of control tDataGridName to tLine1CRData
      --
      put the number of lines of tFileData into tNumberOfLines
      put the number of items of  tLine1Data into tNumberOfColumns
      repeat with tLineNbr = 2 to tNumberOfLines
         put line tLineNbr of tFileData into tLine2Data
         repeat with tItemNbr = 1 to tNumberOfColumns 
            put item tItemNbr of tLine1Data into tColumnName
            put item tItemNbr of tLine2Data into tItemData
            put tItemData into aDataArray[tLineNbr-1][tColumnName]
         end repeat
      end repeat
      set the dgData of group tDataGridName to aDataArray
   end if
   ----------------------------------------------------
   if  tTask = "AddLine"
   then
      ------------------------------
      put line 1 of tFileData into tStringColumnNames
      put line 2 of tFileData into tRowData
      replace "," with cr in tStringColumnNames
      replace "," with tab in tRowData
      put the dgNumberOfLines of group tDataGridName + 1 into tNewLineNbr
      dispatch "AddLine" to group tDataGridName with tRowData, tStringColumnNames, tNewLineNbr
      dispatch "ScrollIndexIntoView" to group tDataGridName with tNewLineNbr
   end if
   ------------------------------
   // put "ID, Colom A,Colom B,Colom C"&lf&"A02,Info UJ,Info UK,Info UL" into tUpdateLineData
   if  tTask = "UpdateLine"
   then
      put line 1 of tFileData into tColumnNames
      put line 2 of tFileData into tLineData
      put item 1 of line 1 of tFileData into tLineColumnID
      put item 1 of line 2 of tFileData into tLineID
      put the dgData of group tDataGridName into aDataArray
      put the keys of aDataArray into tKEYNumbers
      put tKEYNumbers into tOrgKeyNumbers
      replace cr with "," in tKEYNumbers
      put max(tKEYNumbers) into tMaxLineNbr
      put min(tKEYNumbers) into tMinLineNbr
      put the keys of aDataArray[tMinLineNbr] into tArrayColumnNames
      repeat for each line tLineNbr in tOrgKeyNumbers
         put aDataArray[tLineNbr][tLineColumnID] into tRowLineID
         if tRowLineID = tLineID
         then
            put tLineNbr into tLineID2Update
            put the keys of aDataArray[tLineNbr] into tColumnData
            repeat for each item tColumnName in tColumnNames
               put itemOffset(tColumnName,tColumnNames) into tItemPosition
               if tItemPosition > 1
               then
                  put item tItemPosition of tLineData into tItemNewData
                  put tItemNewData into aDataArray[tLineNbr][tColumnName] 
               end if
            end repeat
         end if
      end repeat
      set the dgDataOfIndex[tLineID2Update] of group tDataGridName to aDataArray[tLineID2Update]
   end if
   ------------------------------------------------------
   if  tTask = "DeleteLine"
   then
      put line 1 of tFileData into tColumnNames
      put line 2 of tFileData into tLineData
      put item 1 of line 1 of tFileData into tLineColumnID
      put item 1 of line 2 of tFileData into tLineID
      put the dgData of group tDataGridName into aDataArray
      put the keys of aDataArray into tKEYNumbers
      ----
      repeat for each line tLineNbr in tKEYNumbers
         put aDataArray[tLineNbr][tLineColumnID] into tRowLineID
         if tRowLineID = tLineID
         then
            put tLineNbr into tLine2Delete
         end if
      end repeat
      send "DeleteIndex "& tLine2Delete to the dgControl of group tDataGridName in 0 seconds
   end if
   ------------------------------
end DataGridCreateColumnsFillFromCSV


dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10305
Joined: Wed May 06, 2009 2:28 pm

Re: Simple CSV to DataGrid Command

Post by dunbarx » Wed Mar 12, 2025 7:38 pm

Paul.

If I ever had to load a CSV file into anything at all, wouldn't it be simpler to replace comma with tab and just load? The CR's are still in the right place.

I hear that CSV is sinister, in that comma is far too common a character to reliably parse any sort of typical data, but what could go wrong with that simple one-liner method?

Craig

Klaus
Posts: 14177
Joined: Sat Apr 08, 2006 8:41 am
Contact:

Re: Simple CSV to DataGrid Command

Post by Klaus » Wed Mar 12, 2025 10:43 pm

There is a nice article on Richards website:
https://www.fourthworld.com/embassy/art ... t-die.html
And a lengthy script to parse CSV files with nested COMMAS inside quotes etc...

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10305
Joined: Wed May 06, 2009 2:28 pm

Re: Simple CSV to DataGrid Command

Post by dunbarx » Wed Mar 12, 2025 11:04 pm

Klaus.

Always fun to re-read Richard's brilliant and poignant essay. Everyone should; takes three minutes.

Craig
Last edited by dunbarx on Thu Mar 13, 2025 4:36 pm, edited 1 time in total.

mrcoollion
Posts: 738
Joined: Thu Sep 11, 2014 1:49 pm

Re: Simple CSV to DataGrid Command

Post by mrcoollion » Thu Mar 13, 2025 7:35 am

LOL, love it.

So yes you are all correct.
CSV is not the best way to go but i needed it, and I wanted an easy way to populate an manipulate a DataGrid.
Maybe some one can make a better and more versitile and especially easy to use version ?!
If you do, please post it here?

Regards,

Paul

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10043
Joined: Sat Apr 08, 2006 7:05 am
Contact:

Re: Simple CSV to DataGrid Command

Post by FourthWorld » Thu Mar 13, 2025 11:32 am

Isn't that what setting the dgText of the DataGrid does?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

stam
Posts: 3061
Joined: Sun Jun 04, 2006 9:39 pm

Re: Simple CSV to DataGrid Command

Post by stam » Thu Mar 13, 2025 4:31 pm

FourthWorld wrote:
Thu Mar 13, 2025 11:32 am
Isn't that what setting the dgText of the DataGrid does?
Almost - that's for TSV...

Simplest is to convert CSV to TSV and then set the dgText.
I have a recollection that there is a setting to use the first line as column names (set the dgText[true]?), but even if not it's simple matter of converting the first line to the column names and using lines 2 to -1 for setting the dgText.

The real challenge is usually converting CSV to TSV (or anything else!) as it's truly an appalling format...

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10305
Joined: Wed May 06, 2009 2:28 pm

Re: Simple CSV to DataGrid Command

Post by dunbarx » Thu Mar 13, 2025 4:51 pm

Even if the originators of that mess failed to appreciate its pitfalls, I assume (hope) that any dataSet configured as CSV does NOT ever use commas in any other way. Similarly, if rigorously applied, I guess any character, like "A", would do as well. Tabs are only nice because they are not printable. I never work with such formats, and am trying to imagine a body of text where commas are used not only as a delimiter but additionally as "ordinary" characters.

So then isn't a "good" CSV dataSet changeable into TSV by simply replacing commas with tabs? I assume because of all the hubbub this is not so.

Craig

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10043
Joined: Sat Apr 08, 2006 7:05 am
Contact:

Re: Simple CSV to DataGrid Command

Post by FourthWorld » Thu Mar 13, 2025 5:31 pm

stam wrote:
Thu Mar 13, 2025 4:31 pm
FourthWorld wrote:
Thu Mar 13, 2025 11:32 am
Isn't that what setting the dgText of the DataGrid does?
Almost - that's for TSV...

Simplest is to convert CSV to TSV and then set the dgText.
I have a recollection that there is a setting to use the first line as column names (set the dgText[true]?), but even if not it's simple matter of converting the first line to the column names and using lines 2 to -1 for setting the dgText.

The real challenge is usually converting CSV to TSV (or anything else!) as it's truly an appalling format...
Thanks for the clarification.

I'd just use the CSV to TSV function listed in the "Why CSV Must Die" article, send then let the DG handle the rest.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10043
Joined: Sat Apr 08, 2006 7:05 am
Contact:

Re: Simple CSV to DataGrid Command

Post by FourthWorld » Thu Mar 13, 2025 5:36 pm

dunbarx wrote:
Thu Mar 13, 2025 4:51 pm
So then isn't a "good" CSV dataSet changeable into TSV by simply replacing commas with tabs? I assume because of all the hubbub this is not so.
Take a gander at the article Klaus linked to above.

The problem with using commas as a delimiter is that they often occur in data, requiring that in-data commas be escaped. If the escape characters are also commonly found in data, then the escapes also need to be escaped. Comedy mayhem ensues.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

stam
Posts: 3061
Joined: Sun Jun 04, 2006 9:39 pm

Re: Simple CSV to DataGrid Command

Post by stam » Thu Mar 13, 2025 5:36 pm

dunbarx wrote:
Thu Mar 13, 2025 4:51 pm
So then isn't a "good" CSV dataSet changeable into TSV by simply replacing commas with tabs? I assume because of all the hubbub this is not so.
The difficulty arises from the fact that the comma-delimited fields may themselves contain commas, making it an extremely poor choice for column delimiter and pretty much impossible to just replace commas with tabs.

If the fields do not contain commas then yes, pretty straightforward. Otherwise it usually requires cumbersome workarounds.

If you've ever had to disentangle the mess that CSV data has fields with commas in them entails, you would rather rapidly develop a severe hatred of the format and wonder why on earth every single vendor didn't just default to tab-delimited data...

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10043
Joined: Sat Apr 08, 2006 7:05 am
Contact:

Re: Simple CSV to DataGrid Command

Post by FourthWorld » Thu Mar 13, 2025 6:20 pm

stam wrote:
Thu Mar 13, 2025 5:36 pm
If you've ever had to disentangle the mess that CSV data has fields with commas in them entails, you would rather rapidly develop a severe hatred of the format and wonder why on earth every single vendor didn't just default to tab-delimited data...
CSV feels like a prank submitted in a meeting as a joke, but no one in the room got the joke and just implemented it, and now we're all stuck with it.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

SparkOut
Posts: 2943
Joined: Sun Sep 23, 2007 4:58 pm

Re: Simple CSV to DataGrid Command

Post by SparkOut » Thu Mar 13, 2025 8:43 pm

Even back in the original ASCII days there were field and record separators that could have been used and standardised.
I guess it was too much to convince people that "non printable/non readable" characters were there and meant something significant.
And then the committee took a lunchbreak and ran out of time for the afternoon agenda, so just signed off on csv to show they hadn't been timewasting.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10305
Joined: Wed May 06, 2009 2:28 pm

Re: Simple CSV to DataGrid Command

Post by dunbarx » Thu Mar 13, 2025 9:26 pm

Gentlemen.

I know the pitfalls. That is why I wrote earlier:
Even if the originators of that mess failed to appreciate its pitfalls, I assume (hope) that any dataSet configured as CSV does NOT ever use commas in any other way.
My point was that I assume that any CSV dataSet does NOT and CAN NOT contain commas as part of the actual text, rather only inserted as a delimiter. It must be so, or any processing will fail, no?

And I was the one that recommended as required reading Richard's website after Klaus mentioned it.

Craig

SparkOut
Posts: 2943
Joined: Sun Sep 23, 2007 4:58 pm

Re: Simple CSV to DataGrid Command

Post by SparkOut » Thu Mar 13, 2025 10:37 pm

CSV data can and frequently does contain commas, "fields" which contain commas must be encapsulated, typically with being surrounded by double quotes. There's not even a strict standard of whether quoting fields is necessary within the same dataset if a record does not contain a comma as part of the data. Different importers and exporters from and to csv have different interpretation of what "standards" to adopt.
Quoting fields of course also means that any fields containing double quotes need to have those escaped. And it just continues to get even more messy from there.
Richard's article is eloquent and laudable. But since there is so much insanity about, and we have to deal with such things, he also refers therein to a hugely valuable csvToTab converter which is the most comprehensive way to handle practically every "version" of the worst "standard" of data presentation.

Post Reply