Snowflake cloud Question?

[css_jay99]

Thanks

just tried out DBeaver, certainly a better interface and experience than the default web interface.

There are lots of tables and columns with missing rows, missing parent/child rows or wrong values in columns. Only possible resolution is to identify the missing records which is fine but I then have to rebuild the data by looking at multiple tables and applying some additional business logic ... before finally inserting/ updating. Easy work in oracle.

Client will most likely be using tableau/cognos for the reporting layer but I am not invoved in that side of things.

Looks like javascript may be the way to go.

[LondonManc]

Python's useful but I would imagine they have their own ETL tool functionality. If they don't, they should!

In terms of reporting, depends what your reporting requirements are - very much a case of visualisation vs canned reports. Best tools I've used are Tableau for visualisation and Business Objects for generic reporting and analysis. A lot of bigger companies have different tools for both with varies combinations of Tableau, Power BI, Qlik, etc for visualisation and Cognos/SSRS/BusinessObjects for reporting.

An interesting new contending for the cloud is Chartio: Chartio + Snowflake = Data for All in the Cloud | Chartio Blog . I've never used it but they do a free trial so may be worth a look.

[hairymouse]

I guess another option if you are using blob storage is Data Factory. That would allow you to ingest the files in blob storage, fix them and then output to another blob storage and then load that into Snowflake.

Could you give more detail about what kind of fix you need that can't be done in SQL against a staging table and then selected into the final table?

[hairymouse]

I've used DBeaver to connect to Snowflake and it was way better than the web interface. Free as well.

Snowflake allows you to write stored procedures in javascript if that helps.

[Paralytic]

What does your accountant say?

[css_jay99]

My skillset is primarily on Oracle database and unix, so snowflake is literally very new to me, .... as in a couple of days

I am currently ingesting data from Oracle into snowflake cloud via blob storage. Some of the 30yr old data is afragmented so needs some fixing. This would have been easy within my comfort zone using pl/sql to deal with the issues that can't be resolved with sql. Absolutely no chance of cleansing the data at oracle source so needs to be done at the staging side in snowflake.

Question is what is the best progaming language to use within snowflake for data cleansing?, someone mentioned python but just wanted to be sure that gives just as much flexibility as pl/sql.

Also what is the best 3rd party too to run queries in snowflakes, the web interface is just not my cup of tea

Thanks