Skip to content

tomerr-ma/GCS-To-Big-Query

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

72 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GCS To Big Query Cloud Function

Google Cloud Function for processing json log files in Google Cloud Storage and creating BigQuery tables.

Deploy from gcloud CLI

Open Google Cloud Shell

curl -LO "https://github.com/cloudflare/GCS-To-Big-Query/archive/master.zip" && unzip master.zip && cd GCS-To-Big-Query-master

Update the environment variables in deploy.sh (e.g. nano deploy.sh)

BUCKET_NAME="" # required – The name of Google Cloud Storage bucket used for Cloudflare Logpush logs.
DATASET="" # optional – BigQuery dataset to write to. Will be created if necessary.
TABLE="" # optional – BigQuery table to write to. Will be created if necessary.

Deploy function

sh ./deploy.sh

Note: You can also deploy this function locally using the same steps below. The only requirement is the gcloud SDK.#Test1234

About

Google Cloud Function to push json files from GC Storage to Big Query

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages

  • JavaScript 79.6%
  • Shell 20.4%