Monitoring scans with the Scan Monitor

The Scan Monitor shows, for one table, where every part of a scan stands right now: which fileboxes are scanned, which are still in progress, and which failed and why.

When you upload or submit data, Datarails scans it into the table, then runs the calculation models and Auto Calculate files that depend on it. The Scan Monitor lays out that whole chain on one screen, so you can answer questions like:

  • Is my latest upload already in the table, or still being scanned?
  • Which file or version failed, and what was the reason?
  • Which calculation models and Auto Calculate files are waiting on this scan?
  • Where is most of the time being spent?

Viewing the Scan Monitor does not change your data. Items that failed also offer Rescan, Manual calc or Calculate buttons, which start a new run. The Scan Monitor does not predict when a scan will finish.

Watch this short video to see where to find the Scan Monitor and how to read it:

Scan Monitor overview (1:19). Shown with demo data.

Before you start

You need access to the Tables section to use the Scan Monitor. You see only the tables you own.

  • Who can open it: Admins and table owners. Viewers and Collaborators cannot open the Data flows page.
  • No tables listed: if you don't own any tables, the page shows No tables available. Ask your Datarails admin for Owner access to the table.
  • Availability: the Scan Monitor is being rolled out gradually. If you don't see the Scan monitor option, contact your Customer Success Manager.

Opening the Scan Monitor

The Scan Monitor is a view on the Data flows page, next to the Data flow view. When you open Data flows, the Scan Monitor is the view you see first. You can also open it from:

  • Tables list, Last scan status column: while a table is scanning, click the external-link icon (Open scan monitor).
  • Tables list, row menu: click the three-dot menu on the table row and select Scan monitor.
  • Data flows page header: use the Data flow | Scan monitor toggle. While a scan is running, the Data flow view also shows a progress bar with a Scan monitor link.
  • Excel add-in: click View details in the scan progress dialog or in the submit warning. The Scan Monitor opens in your browser.
Tables list with the Financials table scanning at 41 percent and the Open scan monitor icon next to its progress bar
A table that is still scanning shows a progress bar and the Open scan monitor icon in the Last scan status column.

To switch to another table, choose it from the Table: selector in the header. You stay on the Scan Monitor view.

Note: the Dataflow icon in the Tables list always opens the Data flow view, not the Scan Monitor.

Reading the screen

The Scan Monitor shows the table's scan as three columns of cards, in the order data flows through them:

  1. Input Fileboxes: the fileboxes that load data into the table.
  2. Calculation model: the calculation models that run on the table's data.
  3. Auto calculation: the Auto Calculate fileboxes that recalculate from the results.
The Scan Monitor board for the Financials table: three columns of cards, the Scan progress bar reading 5 of 12 scanned, and the summary panel on the right
The three columns, the Scan progress count in the header, and the summary panel on the right.

Each column header shows how many items it holds, for example Input Fileboxes (12). While a column is still processing, its header also shows its own progress percentage. Drag the board to move around it, and use the zoom control to zoom in or out. Reset view returns to the default zoom.

Scan progress

While a scan is running, the header shows Scan progress: with a bar and a count, for example 14 / 20 Scanned. The count includes every item in the current scan, across all three columns. While Datarails is still gathering the items for the scan, the header shows Setting up.

When the scan ends, the header shows Last scan completed: with the date and time. If you are on the screen when it finishes, a Scan completed window shows whether errors or warnings were found. Click View details to open the activity log.

The screen updates on its own while a scan runs. You don't need to refresh the page.

Statuses

Each card shows its status:

Status What it means
Scan error The item failed. Click the card to see which version failed and why.
Scan warning The item finished, but something needs your attention.
In progress The item is being processed now.
Scanned The item finished successfully.
Pending The item is waiting for an earlier step to finish.
Not started The item has not started yet.

Pending and Not started cards are greyed out and can't be opened yet.

Within each column, cards are grouped by status, with a count on each group, for example 12 SCANNED or 3 NOT STARTED. Click a group header to collapse it. Items that are not part of the current scan are grouped under NOT IN SCAN, which is collapsed by default. After the scan ends, this group is called NOT IN LATEST SCAN.

To focus on some statuses, use Filter: > Status. Cards that don't match are dimmed rather than hidden. To find a specific item, type its name in Search.

Timing

Input filebox and Auto Calculate cards show how long they have been running, or how long they took, for example 1h 2m 3s. Calculation model cards don't show a time.

A brown clock on an input filebox card means a new version was uploaded to it during the scan. This can slow the scan down.

To see where most of the time is being spent, compare the times on the cards, or open a filebox and look at the scan duration of each version.

Summary panel

The panel on the right starts collapsed, showing the main numbers: Duration, Scanned, Errors, Warnings, Not started and Not in scan. Click the arrow to expand it. It has three tabs:

  • Summary: the same numbers, with the scan duration live while the scan runs.
  • Logs: what happened during the scan, newest first. See the next section.
  • Downstream: Auto Calculate files affected by the scan. See Downstream items below.

Viewing item details and errors

Click any card (except Pending or Not started) to open its details panel on the right. Click anywhere on the board, or the ×, to close it.

Filebox details

  • Header: the filebox name and type (Input Filebox or Auto-calc Filebox). The external-link icon opens the filebox in a new tab.
  • VERSIONS: how many versions are scanned (for example, 8 of 10 scanned) and how many have issues. The three most recent versions are listed with their status, date tag and scan duration. A New badge marks a version uploaded while the scan was running. Click All versions to see every version.
  • DEPENDENCIES: the calculation models and Auto Calculate files linked to this filebox. Click the arrow next to an item to jump to its card. Click View connections to see the links as a diagram.
  • GENERAL INFO: the latest tag and the latest submission date.

If the filebox failed, the panel also shows Rescan (Input Filebox) or Manual calc (Auto-calc Filebox).

Calculation model details

The panel shows the model's status and the fileboxes it depends on, plus View connections. The external-link icon opens the calculation model in a new tab. If the model failed, click Calculate to run it again.

All versions

The Versions window lists every version of the filebox, with Version, Date tag, Submission date, Scan duration and Scan status. Sort by any column, filter by status, or search. It is sorted by date tag, newest first.

To rescan, click Rescan and choose one of these:

  • Rescan failed versions: re-runs only the versions that errored. This is the default.
  • Rescan unfinished versions: re-runs everything not yet completed.
  • Rescan all versions: re-runs every version, including ones that scanned successfully.

Failed cards

A failed Input Filebox card shows Rescan and Go to Filebox. A failed Auto-calc Filebox card shows Manual calc and Go to Filebox. Go to Filebox opens the filebox versions page, where you can find and fix the problem version.

Logs and the activity log

The Logs tab lists what happened during the scan, such as Filebox scan started, Filebox scan failed or Auto-calc failed. Each entry shows the time and what triggered it, for example a user, Auto or Version submission.

  • Errors and warnings have an icon. Hover over it to see the reason.
  • Click a filebox or calculation model entry to jump to its card.
  • Use Filters to show only Errors, Warnings or Success, or only actions triggered by User or System.

Click Full activity log to open the full log in a window. It has columns for Timestamp, Action, Type, Name, Triggered by and Description. Select Show only failed logs to focus on errors, or click the download icon to export the log as a CSV file.

Downstream items

Downstream items are Auto Calculate files that are affected by the scan but not mapped to the table. They are processed as part of the scan, and the Downstream tab shows their results.

For each downstream Auto-calc Filebox you can:

  • View connections: see how it links to the table's fileboxes and calculation models. The diagram shows only fileboxes that are part of the current scan.
  • Open filebox in a new tab: open the Auto Calculate file.
  • See an error: if it failed, hover over the error icon and click it (View error in activity log). The activity log opens, filtered to Auto-calc failed.

If no files depend on the scan, the tab shows Nothing downstream.

Scans in the Excel add-in

While data is scanning into a table built from your workbook, the Datarails ribbon in Excel shows Data is scanning (N%). The percentage is for the table that is furthest behind. Viewers and Collaborators do not see this indicator.

Scan progress dialog

Click Data is scanning on the ribbon to open the dialog.

  • One table: the dialog shows <Table> table being scanned and how many files are scanned (for example, 12 of 20 files scanned).
  • Several tables: the dialog shows Scanning N tables into this file, with one section per table. Expand a section to see progress for Input Fileboxes, Calculation models and Auto-calc Filebox.

The dialog refreshes on its own. Values may change until the scan is done.

  • View details: opens the Scan Monitor for the table in your browser. The dialog stays open.
  • Close: closes the dialog. The scan keeps running.

Submitting while a scan is running

If you submit a file while a table it feeds is still scanning, Datarails asks you to confirm first:

Your data is still being scanned. If you submit now your submission could be based on outdated numbers. Are you sure you want to submit?

The window also shows the current scan progress, with a View details link to the Scan Monitor.

  • Submit anyway: submits the file now.
  • Cancel: stops the submit. Nothing is sent.

Closing the window also cancels the submit.

FAQ

How do I know my latest upload is in the table?

Open the filebox card and check VERSIONS. When your version shows as scanned, its data is in the table.

When will the scan finish?

The Scan Monitor does not estimate a finish time. Use the progress count in the header, and compare the scan durations of earlier versions in All versions.

Why can't I open a card?

Cards in Pending or Not started have not run yet, so there are no details to show. They become clickable once they start.

What do the brown clock and the New badge mean?

A new version was uploaded to the filebox while the scan was running. This might affect the scan progress.

A file failed. What should I do?

Open the card and hover over the error in VERSIONS or Logs to see the reason. Fix the version from Go to Filebox, then click Rescan.

I don't see the scan indicator in Excel.

The indicator appears only for tables built from the open workbook, and only while they are scanning. It does not appear for workbooks that only read tables through Datarails functions, or for Viewers and Collaborators.

I clicked Submit anyway. What happens?

Your file is submitted as usual. Because the scan had not finished, the numbers in your submission could be based on outdated data. After the scan completes, refresh your file and check the numbers.




© Datarails Ltd. All rights reserved.

Updated

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request