FinOps Hub v13 + Power BI (Storage report) do not show AWS costs when parquet file is dropped directly under ingestion/

[BoubakriMohamed404]

Description:

With FinOps Hub v13, AWS cost data does not appear in Power BI Storage‑based reports when the AWS parquet file (exported as FOCUS 1.0) is manually copied to the hub storage path:
ingestion/costs/<year>/<month>/awstest/

Azure costs render correctly. AWS costs ignored.

Expected behavior:

Storage‑based Power BI reports should display AWS costs also

Actual behavior:

Juste Azure costs are loaded successfully .

Steps to reproduce

1. Deploy FinOps Hub v13.
2. Export AWS costs as FOCUS 1.0 parquet (outside the hub pipelines).
3. Manually copy the parquet to ingestion/costs/<year>/<month>/awstest/ focusfile and add an empty manifest.json file
4. Open Power BI Storage reports and refresh.
5. Observe AWS data are not loaded .

Environment

FinOps Hub: v13
Power BI: FinOps toolkit Storage‑based reports
Data sources:
Azure data lake gen 2 → hub (OK)
with AWS parquet (FOCUS 1.0) manually placed in ingestion/costs/<year>/<month>/awstest/

Doc link
How to create and update FinOps hubs - Cloud Computing | Microsoft Learn
https://learn.microsoft.com/en-us/cloud-computing/finops/toolkit/hubs/deploy?tabs=azure-portal%2Cadx-dashboard#ingest-from-other-data-sources

Can you please provide me with the correcte steps to use FinOpsHub with AWS data in PowerBI storage report ?

[flanakin]

@BoubakriMohamed404 The one thing that's jumping out to me is that the Costs folder should have an uppercase "C". I don't recall if that's a hard requirement for storage reports, but I know it is for Data Explorer ingestion. At first glance, everything else seems to be correct.

I would recommend renaming the folder to Costs (uppercase "C") and seeing if that works. As a quick workaround, you can also edit this line in the ftk_Storage function to change the casing of the folder name we're checking for:

or (data = "focuscost"                  and "Costs"                   = Text.Split([Folder Path],"/"){containerOffset+1})

Let us know if that works. We can change this to be handle casing issues, but that would be an issue for anyone who later moves to Data Explorer, which is really where you'll want to be, so I'm a bit hesitant to go that route. Either way, let us know.

[BoubakriMohamed404]

Sorry the path is correct Costs is already with (uppercase "C")

the manifest.json file is empty as requested by the doc

the focus1.snappy.parquet is exported with those param . from AWS :

but power BI still load juste Azure Data .

I ran a test by copying another Azure Parquet file for one of my subscriptions to this path (this one is generated automatically by the Azure Data Factory pipeline) and it still loads correctly even after being deleted from its original path (this may confirm that the AWS path added is correct, but the problem comes from the focus1.snappy.parquet file).

[RolandKrummenacher]

Thanks for the detailed repro steps, @BoubakriMohamed404!

Since copying an Azure parquet file to the same path works, the path and folder structure are confirmed correct. The issue is specific to the AWS-exported parquet file. After digging through the code, here are the most likely causes:

1. Parquet metadata date filtering (most likely)

The Power BI report reads parquet metadata statistics to filter files by date range. The ftk_ImpalaToJulianDate function assumes 12-byte INT96 Impala-style timestamps, which is what Azure Cost Management exports use. AWS FOCUS exports likely use INT32 or INT64 date encoding instead. If the metadata parsing produces incorrect dates, the file would be silently excluded by the date filter.

To test: Try setting the Number of Months parameter to blank/empty (to disable date filtering) and refresh. If the AWS data appears, this confirms the metadata parsing is the issue.

2. Power BI Parquet.Document() compatibility

Power BI's parquet reader is more limited than tools like PyArrow. Certain parquet features, compression settings, or encodings used by AWS may cause Parquet.Document() to return 0 rows, which would silently drop the file.

To test: Try opening the AWS .snappy.parquet file directly in Power BI Desktop (Get Data > Parquet) to confirm Power BI can read it at all. Also try exporting from AWS without snappy compression (plain parquet) to rule out compression as a factor.

3. Column name or type differences

If the AWS FOCUS export uses slightly different column names or date formats than what Azure produces, the downstream type conversions could fail for those rows.

To test: Compare the column names in the AWS parquet file (e.g., using Python/PyArrow) against the Azure parquet file to check for any differences.

Could you try the first test (clearing Number of Months) and let us know if that makes a difference?

[BoubakriMohamed404]

Hello @RolandKrummenacher Thank you for your reply. As requested, I updated the “Number of months” parameter to be empty.
I then refreshed the schema and data in Power BI:

First, it loads the table row (364 rows are loaded) in Costs table. I believe that the correct Azure data is loaded, then it starts to indicate the size of the loaded data, which is 4 MB.

Finally, I got an error when loading 245,301 errors without the ability to check exactly the errors type

Is there a clear guide for doing all the integration work with AWS? I don't think I'm the first person to do this anyway.