Data Conversion

Started by Andy Brice on 12/20/2019
Andy Brice 12/20/2019 5:00 pm
I have been working on a data conversion tool. It's not an outliner/PIM etc. But many CRIMPERs must face issues around data conversion as they move data from 1 product to another. So I thought it might be worth giving it a mention here, in case anyone is interested:
https://www.easydatatransform.com/

It is a desktop tool for Windows and Mac, so you don't need to upload your confidential data to any third party servers.

Currently it can input and output Excel and delimited text files (e.g. CSV) and perform 36 different transformations. Hopefully, in the future I will add XML/JSON/SQLite and other formats that people ask for. Maybe even OPML. Although I have to think carefully about the best way to flatten and unflatten tree formats such as XML into a table and back again. Does anyone know any exising tools that do a good job of that?

It has a 7 (non-consecutive) days free trial. After that you have to purchase a subscription.

Happy to answer any questions.

Andy Brice

Alexander Deliyannis 12/20/2019 7:26 pm
Hi, we had briefly discussed this here:
https://www.outlinersoftware.com/topics/viewt/8606/15

But it definitely does deserve a thread of its own!

Aside a brief run at the time, I have not had the chance to try it with real data yet, but it seems to me that you've included additional transformations; looks good!

As noted in that thread, Flowheater is one (and the only) similar programme I am aware of. Re XML currently it can only handle flat files https://flowheater.net/en/documentation/adapter-xml

Interestingly, I currently need a transformation from/to JSON to/from SQL (but even a simple format like CSV will do) and have found nothing of the sort.
Andy Brice 12/20/2019 7:39 pm

Aside a brief run at the time, I have not had the chance to try it with
real data yet, but it seems to me that you've included additional
transformations; looks good!

There has been a lot of additional work and polishing since that previous post.

As noted in that thread, Flowheater is one (and the only) similar
programme I am aware of. Re XML currently it can only handle flat files
https://flowheater.net/en/documentation/adapter-xml

It seems they can convert a CSV to XML, which is pretty straightforward. It is the conversion from XML (or JSON) to CSV that is tricky, especially if it has a deep tree.

I don't know much about Flowheater, but it looks relatively complex. in Easy Data Transform we are trying to come up with something that anyone can get the hang of within a within a few minutes.

Interestingly, I currently need a transformation from/to JSON to/from
SQL (but even a simple format like CSV will do) and have found nothing
of the sort.

I am hoping to tackle XML and JSON soon. But it depends what sort of feedback we get on v1.0.0.
Pierre Paul Landry 12/20/2019 8:08 pm
Andy Brice wrote:
I am hoping to tackle XML and JSON soon. But it depends what sort of feedback we get on v1.0.0.

Hi Andy,

If it helps, I'm using tools from Chilkat to do this in InfoQube and it is easy to use and very robust. Some, including XML and JSON, are actually free !
Check it out !

https://www.chilkatsoft.com/

Pierre
Andy Brice 12/20/2019 9:09 pm

If it helps, I'm using tools from Chilkat to do this in InfoQube and it
is easy to use and very robust. Some, including XML and JSON, are
actually free !
Check it out !

https://www.chilkatsoft.com/

Thanks Pierre, I haven't heard of that library before. Looks interesting. I am using the Qt library and that includes classes for XML and JSON parsing. So parsing isn't too much of a problem. The issue it how to map what is essentially a tree into a table and then possibly back into a tree again at the end. I have some ideas, but I haven't fully thought it through yet. One possibility is to preserve the tree structure in the column names.

Andy Brice
Alexander Deliyannis 12/20/2019 11:27 pm
So one can import JSON files in InfoQube?

If yes, is there a specific extension needed?

Pierre Paul Landry wrote:
If it helps, I'm using tools from Chilkat to do this in InfoQube and it
is easy to use and very robust.
Pierre Paul Landry 12/22/2019 12:51 am
Hi Alexander,

So one can import JSON files in InfoQube?

No, but it could certainly be added if there was a request for it... ;-)

JSON is used internally to store some settings et al and of course when interacting with Google services


Pierre
Andy Brice 6/11/2020 11:21 am
Alexander Deliyannis wrote:
Interestingly, I currently need a transformation from/to JSON to/from
SQL (but even a simple format like CSV will do) and have found nothing
of the sort.

Alexander

The latest version (v1.6.0) is able to both import and export JSON:
https://www.easydatatransform.com/easydatatransform_v160.html

It can also export to SQL:
https://www.easydatatransform.com/convert_csv_to_sql.html
(The example is CSV to SQL, but JSON to SQL would wok just as well).

Andy Brice
Ken 6/11/2020 2:23 pm
Andy Brice wrote:
The latest version (v1.6.0) is able to both import and export JSON:
https://www.easydatatransform.com/easydatatransform_v160.html

It can also export to SQL:
https://www.easydatatransform.com/convert_csv_to_sql.html
(The example is CSV to SQL, but JSON to SQL would wok just as well).

Andy Brice

This is good news as I am possibly looking at using a photo site that exports its data in JSON format, and I was wondering how I would be able to extract what I need.

Thanks for this update,

--Ken
Andy Brice 6/11/2020 4:54 pm
This is good news as I am possibly looking at using a photo site that
exports its data in JSON format, and I was wondering how I would be able
to extract what I need.

Ken

Please let me know if you have any problems.

Andy Brice
Ken 6/12/2020 1:51 am
Andy Brice wrote:
This is good news as I am possibly looking at using a photo site that
>exports its data in JSON format, and I was wondering how I would be
able
>to extract what I need.

Ken

Please let me know if you have any problems.

Andy Brice

Thanks, Andy. I am still a ways out on that project, but I just found out that an alternative also offers JSON format downloads. So now I need to pick a site, load up the images, and then collect the data.

--Ken