❌

Normal view

Google aims to make BigQuery spending on AI functions more predictable with token quotas

15 October 2026 at 12:25

Google has made daily token quotas generally available for BigQuery’s generative AI functions to give enterprises a way to limit AI consumption by project and individual user.

These quotas, which can be managed via the IAM and Admin tab in the Google Cloud Console, cover both input and output tokens at the project and individual-user levels per day and apply to the AI.GENERATE, AI.GENERATE_TABLE, AI.GENERATE_TEXT, AI.CLASSIFY, AI.IF, AI.AGG, and AI.SCORE functions.

Last year in November, the hyperscaler added AI-based SQL functions to its managed data warehouse and analytics platform to help enterprises reduce the complexity of running large-scale analytics, especially on unstructured data.

These functions, which use LLMs to prepare unstructured data for analysis, reduce the traditionally time-consuming process of preparing such data because SQL cannot understand its nuances and meaning.

However, embedding functions such as AI.GENERATE_EMBEDDING, also available as ML.GENERATE_EMBEDDING and AI.EMBED are not subject to these quotas because they run on different, substantially cheaper models, the company wrote in a blog post.

For the other functions, by default, input tokens are capped at 200 billion per project and 150 billion per user per day, while output tokens are capped at 20 billion per project and 15 billion per user per day.

The output token quotas also include the reasoning tokens used by the LLM, the company wrote, adding that enterprises also have the option to raise a request to increase these default quotas.

There is also the option to manage these quotas programmatically through the Cloud Quotas API, the gcloud CLI, and Terraform.

How BigQuery enforces token quotas

Beyond that, however, there are nuances to how the data warehouse platform enforces token quotas.

BigQuery rejects a query with a “QuotaExceeded” error if the token quota it needs has already been exhausted. If the quota runs out while the query is running, BigQuery stops sending new requests to the LLM, the company wrote in its documentation.

That means the results of a query can be affected.

For AI.IF, BigQuery can return partial results if the number of rows it cannot process stays within a specified error limit. If that limit is exceeded, the query fails, the company added.

For AI.GENERATE, though, the query can complete execution, but rows that could not be processed have no generated output and carry an error message in their status field, Google further explained.

To help enterprises avoid hitting these quotas while queries are running, Google has also added a new AI.COUNT_TOKENS function to help enterprise teams estimate token consumption before running an AI function.

However, that function too has limitations. It estimates input tokens only and does not count output or reasoning tokens, additional instructions added by the AI functions, or non-text inputs such as images.

This means the estimate may be lower than the workload’s actual token consumption.

To get a more accurate estimate though, the hyperscaler is recommending enterprise teams to run a small sample workload to measure actual token consumption before setting quotas for larger workloads.

Token quotas offer cost control, but FinOps remains essential

Despite these limitations, Pareekh Jain, principal analyst at Pareekh Consulting, sees the daily token quotas as an “important improvement” for enterprises and their CIOs scaling workloads in BigQuery.

“Earlier, unpredictable token usage could lead to unexpected costs. The new controls help CIOs manage spending, and cost predictability is an important factor in scaling any enterprise workload,” Jain said.

That cost predictability, according to Nhat Bui, deputy director of engineering at Scopic Software, could provide more data professionals with more freedom: “Analysts and data scientists can experiment with less risk of an expensive mistake.”

It also abstracts away the complexity of building mechanisms or dashboards to keep tabs on AI function-related spending in BigQuery, Bui added: “The basic guardrail is now built in: daily limits, usage history, and a function to estimate tokens before running a job. Teams no longer need custom tooling just to avoid a runaway bill.”

However, it does not abstract away the need for FinOps, said Manoj Chandra Jha, principal analyst at Nord-IQ Research.

“For CIOs, BigQuery’s token quotas cap the worst case, not the spending story.  They make it safer to say yes to AI in SQL, but they count tokens rather than dollars and offer no team or business-unit budgets, so finance still can’t see who is spending what,” Jha added.

“The smart play is to treat the quotas as a floor, set real limits, structure projects by team so each becomes a de facto budget line, and layer FinOps reporting on top, because predictability without attribution is only half a governance strategy,” Jha further said.

The bigger challenge for enterprises will be balancing the trade-off between cost control and production reliability, Bui noted.

The lower the token limits, the greater the control over AI spending, but the higher the risk of disrupting production workloads, as queries may fail or return incomplete results when quotas are exhausted before the daily reset, Bui said.

To balance these competing priorities, enterprises should separate production workloads from development and experimental workloads, giving the former more room to operate while keeping tighter limits on the latter, Jha said.

Enterprises should also build pipelines that detect skipped rows and process them again after quotas reset, Jha added.

Snowflake attacker pleads guilty to hack of 165 companies’ data

7 August 2026 at 14:25

A Canadian hacker has admitted being part of a group responsible for several major cyberattacks. Connor Riley Moucka pleaded guilty to being part of a coterie of hackers that hit 165 organizations, resulting in the theft of customer records and the extortion of millions of dollars.

Industry sources have identified Moucka as one of the main players in attacks on data hosted by cloud data warehouse Snowflake. Companies affected by the hacks include the likes of AT&T, Ticketmaster and the Neiman Marcus Group.

He worked with two other hackers: John Edward Binns and Cameron John Wagenius. Binns was not in US custody as of April 2026, while Wagenius, going by the name of Kiberphant0m, was arrested in January 2025 and pleaded guilty in July that year

Moucka and other members of the group used stolen login credentials to compromise data belonging to at least 165 customers of a US-based software-as-a-service company. This unauthorized access was used to steal billions of sensitive customer records and download terabytes of information,

“Connor Moucka hacked over 150 companies and organizations, obtained extremely sensitive information, and extorted the victims for millions of dollars. Today’s guilty plea serves as a reminder to all cybercriminals, regardless of where they live, that they cannot hide behind a wall of anonymity. You will be found and brought to justice,” said assistant attorney general A. Tysen Duva of the Justice Department’s Criminal Division

The trial is the result of a coordinated worldwide action against the Snowflake group. The investigation was led by the FBI but benefited from contributions from the Royal Canadian Mounted Police, the Australian Federal Police, Spain’s Guardia Civil, the Security Service of Ukraine and the Turkish National Police.

This article first appeared on CSO.

New Databricks tool uses AI agents to rewrite legacy SQL at scale

30 July 2026 at 12:37

Databricks is adding an agentic code conversion capability to its Lakebridge toolkit, using Genie Code to provide its new customers another way to move from rival data warehouses to its lakehouse built atop Databricks SQL.

The new capability, called the agentic code converter, uses AI subagents to analyze source code, convert files in parallel, validate the translated SQL, and iteratively retry failed sections within a migration project, while allowing developers to define reusable migration rules for enterprise-specific SQL constructs.

The Lakebridge migration tooling already includes multiple transpilation engines, including the pattern-based BladeBridge technology Databricks acquired in 2025, the compiler-based Morpheus, and the LLM-powered Switch transpiler.

The key difference between the transpilers and the new agentic code converter, according to analysts, lies in how each of them executes conversion and how that could benefit enterprises.

“Deterministic transpilers (Morpheus) handle syntax mapping well: date functions, join syntax, straightforward pattern-to-pattern translation. They get brittle fast on anything requiring control-flow reasoning, cursors, dynamic SQL assembled at runtime, recursive CTEs implemented differently across T-SQL, Teradata, and Oracle,” said Amit Chandak, chief analytics officer at IT consulting firm Kanerika.

“That’s the same 10% to 15% of a legacy codebase that eats most of the manual remediation time on every migration I’ve worked on,” Chandak noted.

In contrast, agentic code conversion, said Pareekh Jain, principal analyst at Pareekh Consulting, is better suited for stored procedures, dynamic SQL, proprietary functions, and complex business logic that require contextual understanding.

“It should improve first-pass conversion quality and reduce manual remediation, especially for the most complex portions of a migration,” Jain noted.

For CIOs, that translates into shorter timelines and lower costs to complete migration projects, especially those that involve thousands of SQL scripts, Jain added.

Chandak, however, said enterprises should temper expectations, mostly because the agentic code converter capability continues to be in beta.

The current beta limits conversion jobs to batches of 300 files, meaning migrations involving thousands of stored procedures still require multiple runs and manual checkpoints, Chandak pointed out.

That limitation could be somewhat offset by the new custom migration skills feature, included with the capability, which allows developers to codify one-off fixes as reusable conversion rules, although whether the agentic system can perform consistently remains an open question till the product matures, Chandak added.

Even if the capability delivers on that promise, Chandak further pointed out that enterprises should not expect it to dramatically shorten end-to-end migration projects because SQL conversion is only one part of the overall modernization effort, as enterprises still need to reconcile migrated data against production systems, sequence dependencies, and obtain business sign-off before workloads can be cut over.

“Because Databricks’ roadmap places data migration and reconciliation in a future phase, the new capability is likely to compress the SQL rewrite effort rather than the overall migration timeline,” Chandak noted.

Databricks isn’t the only data platform vendor using AI to lower the barriers to migration.

Rival Snowflake already has a similar tool, named SnowCovert, that uses agents to help enterprises move their data, data warehouses, business intelligence (BI) reports, and code to Snowflake’s platform.

Google Cloud, too, enables automated migration workflow through Gemini-powered schema and code conversion capabilities in its Database Migration Service.

❌