Google Cloud DNS integration

Use the Google Cloud DNS integration when your domain records are managed in Google Cloud DNS and you want Lightsage to track AI agent visits without installing application middleware. Google Cloud DNS manages the records that send traffic to your site. To collect agent analytics, route those DNS records to a Google external Application Load Balancer first. The load balancer produces the HTTP request logs that Lightsage receives through Cloud Logging and Pub/Sub.

Google Cloud DNS alone does not produce request logs. DNS records do not include request paths, user agents, status codes, or client IP addresses. This setup uses Google Cloud DNS for routing and Google Cloud Load Balancing for request logging.

How it works

Visitor
  -> Google Cloud DNS
  -> Google external Application Load Balancer
  -> Your origin, such as Webflow

Google load balancer request logs
  -> Cloud Logging sink
  -> Pub/Sub topic
  -> Pub/Sub push subscription
  -> Lightsage Agent Analytics

Lightsage only requires the load balancer request logs. Google Cloud CDN is optional and covered in Cloud CDN below.

Before you begin

You need:

Access to the GCP project that manages the domain’s Cloud DNS zone
Permission to edit Cloud DNS records, load balancers, Cloud Logging sinks, Pub/Sub topics, Pub/Sub subscriptions, and service accounts
An origin that Google can proxy to, such as Webflow, a docs platform, or your own web server
Access to Agent Analytics > Integrations > Google Cloud DNS in Lightsage

Start with a test subdomain, such as agent-test.example.com, before moving production traffic.

Configure the integration

Open the Google Cloud DNS integration in Lightsage

In Lightsage, go to Agent Analytics > Integrations > Google Cloud DNS.Copy the Push endpoint shown in the setup screen. You will use it as the Pub/Sub push endpoint after you create the log export.

Create a Google external Application Load Balancer

In the GCP Console, create a Google external Application Load Balancer for the domain or test subdomain you want to track.Configure the load balancer with:

A frontend for the tracked hostname
A managed certificate or certificate that covers the tracked hostname
A backend that points to your origin

You enable backend logging in a later step.Because your origin lives outside Google Cloud, the backend must point to an internet network endpoint group (internet NEG) rather than a Google Cloud instance group. Create an internet NEG with an INTERNET_FQDN_PORT endpoint for your origin hostname, such as your-site.webflow.io:443, and attach it to the backend service.

Internet NEGs require the Premium network service tier. On a global external Application Load Balancer, the internet NEG holds a single endpoint and cannot be combined with other backend types. When you use an INTERNET_FQDN_PORT endpoint, Google validates the origin’s certificate against public CAs, so the origin must serve a valid publicly trusted certificate.

For Webflow, point the internet NEG at the Webflow-hosted origin and validate the setup on a test subdomain before changing production DNS.

Point Google Cloud DNS to the load balancer

In Cloud DNS, update the tracked hostname so it points to the load balancer frontend instead of pointing directly to your origin.The load balancer frontend uses a global anycast IP address, so point the tracked hostname at that IP with an A record, whether it is an apex domain or a subdomain:

example.com.       A  LOAD_BALANCER_IP
docs.example.com.  A  LOAD_BALANCER_IP

Wait for DNS propagation, then verify the hostname serves the same site through the load balancer.

Enable load balancer logging

In the load balancer backend service, enable logging and set the sample rate to 1.0 while validating the integration.A sample rate of 1.0 exports every request handled by the load balancer. You can lower it later if your organization wants to reduce log volume, but Lightsage will then only receive the sampled fraction of requests.

Create a Pub/Sub topic

In the GCP Console, open Pub/Sub > Topics.Click Create topic and use a clear name, such as:

lightsage-agent-analytics-logs

Keep the topic in the same project as the load balancer logs.

Create a push service account

Create or select a service account for authenticated Pub/Sub delivery.A recommended name is:

lightsage-agent-analytics-push

Copy the service account email. You will paste this email into Lightsage so Lightsage can verify that push requests came from your GCP project.

If Pub/Sub cannot sign push requests with this account, grant the Pub/Sub service agent the Service Account Token Creator role on the push service account.

Create a Pub/Sub push subscription

Open Pub/Sub > Subscriptions and click Create subscription.Configure the subscription:

Field	Value
Topic	Select the topic you created
Delivery type	`Push`
Endpoint URL	Paste the Lightsage Push endpoint
Authentication	Enabled
Service account	Select the push service account
Audience	Leave blank unless Lightsage shows an audience value
Payload unwrapping	Enabled

Keep retry and retention settings at your organization’s defaults unless you need a specific retention policy.

Add a subscription transform

In the subscription configuration, click Add a transform.Use this function name:

lightsage_agent_analytics_reduce

Paste this transform:

function lightsage_agent_analytics_reduce(message, metadata) {
  const data = JSON.parse(message.data);

  delete data.insertId;
  delete data.jsonPayload;
  delete data.logName;
  delete data.receiveTimestamp;
  delete data.resource;
  delete data.severity;
  delete data.spanId;
  delete data.trace;

  message.data = JSON.stringify(data);

  return message;
}

Click Validate. The transform removes fields Lightsage does not need before the message is delivered.

Create a Cloud Logging sink

In the GCP Console, open Logging > Log Router.Click Create sink. Give the sink a name, such as:

lightsage-agent-analytics

For Sink destination, choose Cloud Pub/Sub topic and select the topic you created.

After you create the sink, grant its writer identity the Pub/Sub Publisher role (roles/pubsub.publisher) on the destination topic. Cloud Logging configures this automatically only when you create the sink from the console with Owner access to the topic’s project. Otherwise, copy the writer identity email from the sink details and grant the role manually. Without this permission, logs route into the sink but never reach the topic, and no error is shown.

Filter the sink to your load balancer

In Choose logs to include in sink, add a filter for your load balancer.Replace the placeholder values with your forwarding rule and URL map names:

resource.type="http_load_balancer"
resource.labels.forwarding_rule_name="MY-FORWARDING-RULE-NAME"
resource.labels.url_map_name="MY-URL-MAP-NAME"

If you export logs from multiple load balancers, include each pair explicitly:

resource.type="http_load_balancer"
AND (
  (
    resource.labels.forwarding_rule_name="MY-FORWARDING-RULE-NAME"
    AND resource.labels.url_map_name="MY-URL-MAP-NAME"
  )
  OR
  (
    resource.labels.forwarding_rule_name="MY-FORWARDING-RULE-NAME-2"
    AND resource.labels.url_map_name="MY-URL-MAP-NAME-2"
  )
)

Click Create sink.

Save the service account in Lightsage

Return to Agent Analytics > Integrations > Google Cloud DNS in Lightsage.Paste the push service account email into GCP service account and click Save.Lightsage uses this email to verify authenticated Pub/Sub push requests.

Verify log delivery

Send a test request through the tracked hostname:

curl -A "ClaudeBot/1.0" https://docs.example.com/

Check Agent Analytics after a few minutes. The tracked domain should appear after Lightsage receives the first matching request log.

Webflow domains

For Webflow sites, the common starting point is:

Google Cloud DNS -> Webflow

That setup does not produce Google HTTP request logs. To use this integration, update the path to:

Google Cloud DNS -> Google external Application Load Balancer -> Webflow

Validate the load balancer on a test subdomain before moving production traffic. Confirm that pages, assets, redirects, forms, cookies, and canonical-domain behavior still work.

Cloud CDN

Cloud CDN is not required for this integration. If you do not already use a CDN, leave it disabled; the load balancer still logs every request, which is all Lightsage needs. You can enable Cloud CDN on the load balancer later if you want Google to cache content at the edge. Cache hits are still logged by the load balancer, so agent analytics keeps working.

Troubleshooting

Symptom	Check
The site works, but no logs appear	Confirm traffic reaches the Google load balancer, not the old origin DNS record.
Logs appear in Cloud Logging but not Lightsage	Confirm the Log Router sink destination is the Pub/Sub topic used by the push subscription, and that the sink’s writer identity has the Pub/Sub Publisher role on that topic.
Pub/Sub push requests are rejected	Confirm the service account email in Lightsage matches the service account used by the push subscription.
Only some requests appear	Confirm backend logging sample rate is `1.0`.
Webflow pages fail behind the load balancer	Validate host, TLS, redirect, and canonical-domain behavior before switching production DNS.

After Cloud DNS points to the load balancer and the sink and push subscription are active, Google request logs should appear in Lightsage Agent Analytics within a few minutes.

Quickstart

Workflows

Agent Analytics

Integrations

Lightsage Docs Agent

Google Cloud DNS integration

How it works

Before you begin

Configure the integration

Webflow domains

Cloud CDN

Troubleshooting

​How it works

​Before you begin

​Configure the integration

​Webflow domains

​Cloud CDN

​Troubleshooting

How it works

Before you begin

Configure the integration

Webflow domains

Cloud CDN

Troubleshooting