Image for post
Image for post
Photo by Barth Bailey on Unsplash

There are two different methods to synchronize data from an AWS S3 bucket to a GCP Cloud Storage bucket. The first one can be named PULL with which we consider all mechanisms that are triggered by GCP (the target system) to synchronize data, this requires the target system to schedule a job in order to maintain a certain level of consistency between the two systems. …


Image for post
Image for post
Photo by Maximilian Weisbecker on Unsplash

I want to launch specific tasks programmatically on Kubernetes using Kubernetes Job.

How can I launch on-demand jobs on Kubernetes?

In this tutorial, I define a simple Kubernetes Job and then show two ways to make it “on-demand”.

These are the steps we will go through:

  1. Define the Job
  2. Working with CronJob
  3. More versatility with a Python app
  4. Deploy of the Python app

Before we start

Before we start, notice that I’m using Minikube on Windows, a Kubernetes engine that can run locally on my PC. …


Image for post
Image for post
Photo by Erwan Hesry on Unsplash

Docker helps developers create apps removing a lot of headaches about platform compatibility and dependencies.

What I will show in this simple tutorial is the creation of a Docker application similar to the one available here. The main difference is that my version uses a PostgreSQL database instead of a Redis one.

The prerequisite of the tutorial is to have Docker Engine and Docker Compose installed on your machine.

The full repository of this tutorial is available here.

Setup

First thing you have to do is create a project folder (and the folder we will need later). …

About

Stefano Passador

Degree in Computer Science - Master in Big Data Engineer - IT Enthusiast - Milan, Italy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store