So you want your AI agent to read your email and know your schedule. Yeah, you do. Once you set this up you'll wonder how you ever functioned without it.

Here's what I did today: connected all 3 of my Gmail accounts (AppSumo, personal, consulting) plus their calendars to Ace. Now every morning I get a briefing that actually knows what's going on. Urgent emails get flagged in real time. Meeting prep happens automatically because Ace can see what's on the calendar and pull relevant email threads.

It's not hard to set up but there are a few things that'll trip you up if nobody tells you.

The OAuth Setup

Go to Google Cloud Console. Create a project (or use an existing one). Enable the Gmail API, Google Calendar API, and Google Drive API.

Now here's the important part: when you create your OAuth credentials, pick Desktop app as the application type. Not web app. Desktop app. Web app credentials require a redirect URI on a server you're hosting and it's a whole thing. Desktop app gives you a client ID and secret that works with a local auth flow. Way simpler.

Download the credentials JSON. You'll need it for the auth flow.

Scopes

Keep it tight. Here's what I use:

  • gmail.readonly - read emails, search, list threads
  • gmail.send - send emails (optional, but useful)
  • calendar.readonly - see all your events
  • drive.readonly - access Google Drive files

Notice a pattern? Read only wherever possible. This is a safety principle I feel strongly about. Your AI agent should be able to look at everything but only touch what you explicitly allow. gmail.send is the one exception and even that I'm cautious with. I don't let Ace send emails without asking me first.

Multi-Account Token Storage

If you have multiple Google accounts (and who doesn't), you need separate tokens for each one. I store them like:

tokens/
  personal-gmail-token.json
  appsumo-gmail-token.json
  consulting-gmail-token.json

Each token file has the access token, refresh token, and expiry. The refresh token is what matters. Access tokens expire every hour but the refresh token lets you get new ones without re-authing.

Run the auth flow once per account. It'll open a browser, you log in, grant permissions, and the token gets saved. After that it's automatic.

The Test Users Gotcha

This one got me. If your Google Cloud project is in "testing" mode (which it is by default, and you should leave it there unless you're building a real product), only "test users" can authorize. You have to manually add each email address you want to connect in the OAuth consent screen settings.

Go to APIs & Services > OAuth consent screen > Test users > Add users. Put in every email you want to connect. If you skip this step you'll get a cryptic "access denied" error during the auth flow and waste 30 minutes debugging it.

You don't need to publish your app. Testing mode is fine for personal use. Publishing means Google reviews your app and that's a whole process you don't need.

What You Can Build With This

Once your agent has Gmail and Calendar access the possibilities are stupid good:

Daily briefings. Every morning Ace scans all 3 inboxes, pulls unread count, flags anything that looks urgent, summarizes the important threads, and tells me what's on the calendar. I get this on WhatsApp before I've had coffee.

Urgent email alerts. Certain senders or keywords trigger immediate notifications. If a client emails about something broken, I know within minutes, not hours.

Meeting prep. Before any calendar event, Ace can pull recent email threads with the attendees, find relevant docs, and give me a quick brief. "You're meeting with Sarah at 2pm. Last email thread was about the Q1 budget. She had concerns about the contractor line item."

Cross-account visibility. This is the real unlock. I have work stuff in one inbox, personal in another, side project in a third. Ace sees all of them. No more context switching between accounts to figure out what needs attention.

The whole setup took maybe 45 minutes including debugging the test users thing. Now it runs 24/7 on the Mac Mini and I never think about it. This is what an AI agent should be doing. Not chatting about philosophy. Reading your email and telling you what matters.