bitScry

  • Home
  • About
  • Blog
  • Contact

Google

About this site

This is a blog to help me remember some of the coding challenges I’ve faced and hopefully help others out if they’re suffering the same problems.

Recent Posts
  • Cloning a Private Repository in a GitHub Workflow
  • Installing Multiple .NET Versions in GitHub Actions
  • Serving Images from Blob Storage in an Azure Function
  • Redis Caching in .NET
  • Passing Environment Variables From Docker to .NET Web App
Tag Cloud

.NET .NET Core AAD API ASP.NET Core Authentication Azure Azure Active Directory Azure Function Azure Storage Blob Bootstrap C# Console Application CSS DataTable Dependency Injection Dictionary Entity Framework Enum Extension Google GZip HTML Image JavaScript jQuery JSON List Middleware MVC Newtonsoft RestSharp Serilog SQL SSDT SSIS Stored Procedure Stream Table Table Storage Test Web App Web Application WordPress

Programming

Data Transfer – SQL to Google Cloud Storage

Similar to how I’m uploading data to an SFTP the below example streams a SQL table directly to a Google Cloud Storage object. Full code for this is here. Usage Here the StreamWriter needs to be flushed before the upload to GCM is initiated as otherwise the StorageClient will finish Read more…

By Shinigami, 4 years31 December 2019 ago
Programming

Data Transfer – Google Cloud Storage to Azure Blob Storage

Moving files from Google Cloud Storage to Azure Blob Storage is pretty simple as they can just be streamed directly. My source file in Google Cloud Storage was a compressed gzip file which I needed to be unzipped at the Azure Blob Storage destination. Usage

By Shinigami, 4 years30 December 2019 ago
Uncategorised

Data Transfer

As I seem to have been shuffling a lot of data about recently I thought I’d try to document as much of it as possible. Google BigQuery to Google Cloud Storage Google Cloud Storage to SQL SQL to Google Cloud Storage Google Cloud Storage to Azure Blob Storage SQL to Read more…

By Shinigami, 4 years30 December 2019 ago
Programming

Data Transfer – Google BigQuery to Google Cloud Storage

There is a variety of ways of getting data out of BigQuery using the API including streaming the results into memory and creating a file for download, however if you need to download a large amount of data then this will initially need to be exported to Google Cloud Storage Read more…

By Shinigami, 4 years27 December 2019 ago
  • Facebook
  • Twitter
  • Instagram
  • Email
Hestia | Developed by ThemeIsle