eat some code

Google-cloud & Heroku

Using Google-cloud in Heroku with Django

September 2020 #Django  #API  #Heroku  #Google cloud 

I have just setup a Django website on Heroku. That was fairly easy. One of the "gotcha" was the Google-cloud authentication. Getting access to Google translate took a bit more work than other APIs. Here is how I've done it.

Regular Google-cloud setup

To use Google translate, I've used the official python library: google-cloud-translate. It's specified in the documentation to use a JSON file for authentication:

$ export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"

That's easily done locally but I wasn't sure how to do this via Heroku without having to add my keyfile.json under my repository (which I definitely do not want to do).

Every other authentication details reside under the config vars in Heroku. This is were I want to store my Google translate details too. To achieve this, we must use some regular Django settings instead of keyfile.json.

Google-cloud authentication without keyfile.json

Locally, that was really easy. I just had to change:

translate_client = google_translate.Client.from_service_account_json(
    settings.GOOGLE_TRANSLATE_SERVICE_ACCOUNT_FILE
)

into:

from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_info(
    settings.GOOGLE_TRANSLATE_LOGIN
)

translate_client = google_translate.Client(credentials=credentials)

where GOOGLE_TRANSLATE_LOGIN simply contains the same as keyfile.json:

GOOGLE_TRANSLATE_CLIENT_ID = "***"
GOOGLE_TRANSLATE_PROJECT_ID = "****"
GOOGLE_TRANSLATE_PRIVATE_KEY_ID = "****"
GOOGLE_TRANSLATE_PRIVATE_KEY = "-----BEGIN PRIVATE KEY-----\n***********\n-----END PRIVATE KEY-----\n"

GOOGLE_TRANSLATE_LOGIN = {
    "type": "service_account",
    "project_id": GOOGLE_TRANSLATE_PROJECT_ID,
    "private_key_id": GOOGLE_TRANSLATE_PRIVATE_KEY_ID,
    "private_key": GOOGLE_TRANSLATE_PRIVATE_KEY,
    "client_email": f"""google-translate-service-accou@{GOOGLE_TRANSLATE_PROJECT_ID}.iam.gserviceaccount.com""",
    "client_id": GOOGLE_TRANSLATE_CLIENT_ID,
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": f"""https://www.googleapis.com/robot/v1/metadata/x509/google-translate-service-accou%40{GOOGLE_TRANSLATE_PROJECT_ID}.iam.gserviceaccount.com""",
}

Plug into Heroku

With the above working locally, deploying to Heroku should be a piece of cake. All we have to do is to add the corresponding settings (GOOGLE_TRANSLATE_CLIENT_ID, etc.) as config vars on the Heroku website and read then via os.environ.get(...). With the right config vars, everything should work in Heroku too. In your main settings file add GOOGLE_TRANSLATE_LOGIN settings and deploy:

#################
# Heroku settings
django_heroku.settings(locals())

# ....

GOOGLE_TRANSLATE_LOGIN = {
    "type": "service_account",
    "project_id": os.environ.get("GOOGLE_TRANSLATE_PROJECT_ID"),
    "private_key_id": os.environ.get("GOOGLE_TRANSLATE_PRIVATE_KEY_ID"),
    "private_key": os.environ.get("GOOGLE_TRANSLATE_PRIVATE_KEY"),
    "client_email": f"""google-translate-service-accou@{os.environ.get("GOOGLE_TRANSLATE_PROJECT_ID")}.iam.gserviceaccount.com""",
    "client_id": os.environ.get("GOOGLE_TRANSLATE_CLIENT_ID"),
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": f"""https://www.googleapis.com/robot/v1/metadata/x509/google-translate-service-accou%40{os.environ.get("GOOGLE_TRANSLATE_PROJECT_ID")}.iam.gserviceaccount.com""",
}

Heroku "gotcha"

Unfortunately, at this stage, this will not work. With the above, you will get an authentication error. Fixing this took me multiple trials & errors as I could not figure out what was wrong. Obviously, the first thing to do would be to use the Django shell in Heroku and inspect your settings. The settings should match what you have locally. Even if your settings look correct, you might still not be able to authenticate with Google-cloud. Here is the trick: in GOOGLE_TRANSLATE_PRIVATE_KEY, you need to replace all the \n characters in with actual return to the line ! Once you've done that, the authentication will work.

Conclusion

If you are like me and dislike configuring servers, Heroku will save a lot of time but you might still run into weird issues. Hopefully, this post has saved you some time and sanity.