• Visitors can check out the Forum FAQ by clicking this link. You have to register before you can post: click the REGISTER link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. View our Forum Privacy Policy.
  • Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!

Snowflake cloud Question?

Collapse
X
  •  
  • Filter
  • Time
  • Show
Clear All
new posts

    Snowflake cloud Question?

    My skillset is primarily on Oracle database and unix, so snowflake is literally very new to me, .... as in a couple of days

    I am currently ingesting data from Oracle into snowflake cloud via blob storage. Some of the 30yr old data is afragmented so needs some fixing. This would have been easy within my comfort zone using pl/sql to deal with the issues that can't be resolved with sql. Absolutely no chance of cleansing the data at oracle source so needs to be done at the staging side in snowflake.

    Question is what is the best progaming language to use within snowflake for data cleansing?, someone mentioned python but just wanted to be sure that gives just as much flexibility as pl/sql.

    Also what is the best 3rd party too to run queries in snowflakes, the web interface is just not my cup of tea

    Thanks

    #2
    What does your accountant say?

    Comment


      #3
      I've used DBeaver to connect to Snowflake and it was way better than the web interface. Free as well.

      Snowflake allows you to write stored procedures in javascript if that helps.

      Comment


        #4
        I guess another option if you are using blob storage is Data Factory. That would allow you to ingest the files in blob storage, fix them and then output to another blob storage and then load that into Snowflake.

        Could you give more detail about what kind of fix you need that can't be done in SQL against a staging table and then selected into the final table?

        Comment


          #5
          Python's useful but I would imagine they have their own ETL tool functionality. If they don't, they should!

          In terms of reporting, depends what your reporting requirements are - very much a case of visualisation vs canned reports. Best tools I've used are Tableau for visualisation and Business Objects for generic reporting and analysis. A lot of bigger companies have different tools for both with varies combinations of Tableau, Power BI, Qlik, etc for visualisation and Cognos/SSRS/BusinessObjects for reporting.

          An interesting new contending for the cloud is Chartio: Chartio + Snowflake = Data for All in the Cloud | Chartio Blog . I've never used it but they do a free trial so may be worth a look.
          The greatest trick the devil ever pulled was convincing the world that he didn't exist

          Comment


            #6
            Thanks

            just tried out DBeaver, certainly a better interface and experience than the default web interface.

            There are lots of tables and columns with missing rows, missing parent/child rows or wrong values in columns. Only possible resolution is to identify the missing records which is fine but I then have to rebuild the data by looking at multiple tables and applying some additional business logic ... before finally inserting/ updating. Easy work in oracle.

            Client will most likely be using tableau/cognos for the reporting layer but I am not invoved in that side of things.

            Looks like javascript may be the way to go.

            Comment

            Working...
            X