Plan: Available: All plans
Users: Not Available: All users, User Group Permissions Available: FYI Admins
Practice Management Source: Available: All sources
Bulk Export AWS can be used to export all or part of the documents stored in FYI, including key metadata, to a designated Amazon Web Services (AWS) S3 Directory. Bulk Export AWS can be used in the event of a practice split, or in the rare event that your practice chooses to leave FYI.
Documents will be exported in a well-structured format, allowing practices to specify the export of documents stored in a particular Cabinet, associated with a specific Client, or related to a designated Partner. Documents exported to AWS are copies only and are not removed from the FYI platform.
Your data is securely stored in a private S3 directory, allowing you to connect and download files at any time. A separate AWS account or server is not required.
Important Note: Bulk Export is only available on demand. Refer to Before you Begin for more information.
If you experience any issues during the export process, refer to Issues when bulk exporting using Cyberduck.
Understanding Bulk Export AWS
Before you Begin
Ensure that you understand the following key information regarding a Bulk Export AWS:
- The Export is only available to FYI Admins.
- To run a Bulk Export, you must contact the FYI Support Team to enable this function.
- Export documents are a copy of those in FYI, the existing documents are not removed or altered by performing a bulk export.
- Bulk Exports are only performed overnight between 6 pm - 6 am (depending on your location). Any processes that are still running at 6 am will be completed; all other requests will be delayed until 6 pm.
Note: We recommend scheduling your export to avoid our weekly software release, typically on a Thursday evening, as it may restart the export process and cause delays. - The Export list is finalised when a user clicks 'Create' in the Export wizard. Any changes to documents such as creating a new document or deleting a document, or changes to Client Lists, will not be included in the Export.
- If a Category contains a backslash (\) the export converts this to an underscore (_). For example, if the category name is "GST\BAS" this is converted to "GST_BAS".
- If documents have already been exported and you need to change the structure, this will not affect the folders in the previous export. Every export is saved to a separate folder and uses the structure as set when the export is run. The folder name of each export is set as the date and time when the export request was started.
Please be aware of the following items included and excluded from the Bulk Export:
| Included | Excluded |
| Current version of a document | Previous versions of documents |
| Document Threads, including Emails (optional) | Draft Filed Documents and Emails |
| Archived Clients and Documents | Deleted Documents |
| Web Links (.dat file extension) | Document Comments and Tasks |
| Phone Calls, File Notes, and Meetings (.html file extension) | Email attachments not filed separately in FYI. Attachments will be accessed directly from the email. A paperclip icon will display against the imported email, in the list view. |
| Keywords | Document Templates |
| To, From and CC Information recorded against an Email. | Tags |
| Invoices | |
| Document Workflow | |
| Document Activity |
Security
Public and private keys are generated for you by FYI. Access to the exported files is secured by the key pair.
- The public key is held for you by FYI.
- The private key is linked to a username, which is your email address.
The private key is generated as a .pem file and downloaded as part of the export process.
The Export security is controlled by an SFTP client (SSH File Transfer Protocol/Secure File Transfer Protocol). We recommend using Cyberduck for this. Another SFTP client can be used if you are familiar with SFTP clients or have one installed already.
Note: Our support may be limited if you choose to use another SFTP client other than Cyberduck.
Verifying Documents before Running an Export (Optional)
If you want to export documents for a specific Partner, Cabinet or Client, before running a Bulk Export verify the following:
This exports documents for clients where the selected user is held as the Partner for the Client. To export documents for a specific Partner, verify which documents will be included by:
- Add the Partner column to both the Clients list and Documents list.
- Review the entries before initiating the export.
- If changes need to be made, Bulk Update the Partner Field on Clients or Documents list.
You can choose to export from a specific Cabinet or all Cabinets. If you are not exporting all documents or filtering by Partner, review the documents are filed to the relevant Cabinet:
- In the Documents list, filter the Cabinet column.
- Review the documents are filed to the correct cabinet.
- If changes need to be made, move any documents using Bulk Update.
You can choose to export documents for a specific client by running a full export, and then searching their name in dropdown.
Where multiple client documents need to be exported, running a full export (no client selected) and then download only the clients you need using your SFTP client. To do this:
- Run the Export
- Open the exported folders in your SFTP client.
- Hold the Ctrl key (or Cmd on Mac) to select multiple client folders.
- Refer to the SFTP instructions below for more details.
Running the Export
Running an export is completed in three main steps:
- Configure and run the export to generate the files in AWS.
- Use Cyberduck to connect to the SFTP location and download the exported files.
- Review the Document Export CSV files to confirm that the documents were exported and identify any errors.
Configure and Run the Export
When running the Export the first time, you download the private key, the .pem file.
- From FYI, click Settings in the top right-hand corner.
- Select Practice settings from the drop-down, then select Documents.
- Go to the Export tab.
- Click Run. The Export Settings displays.
- Ensure the Medium is set to "AWS Transfer".
- Move to Step 2 - Select Export Criteria.
Important Note: Where no filters are set for Partner, Client or Cabinet, a full export will be performed.
-
From the Export Settings window, set the relevant criteria where required:
Export Criteria Details Client Select a Client from the drop-down to export all documents for that Client only. Partner Selecting a Partner exports documents for clients where the selected user is held as the Partner for the Client. Cabinet You can also select the Cabinet that holds the documents you want to export.
To export all Cabinets leave this as "No option selected".
Incremental When ticked, this exports only documents since the export was last run, not the entire document list.
When Incremental is enabled:
- The export will only include documents that have changed since the last export was run, such as documents that were created, modified, or updated.
- If you are running a Bulk Export via AWS with Incremental enabled and notice a discrepancy between the exported document count and the number shown in the Documents list, this may be because some documents were already exported in a previous run and have not changed since then.
For example, migrated documents that have not changed since the previous export will not be included in the incremental export, potentially affecting the final document count.
Include FYI reference When ticked, all the documents that are exported have the FYI Reference Number appended to the end of the document name.
The index.csv file automatically created includes a column for FYI Reference Number. Refer to Document Export CSV.
If you need to keep a record of any additional metadata you want to keep as a record you can include the FYI Reference number. Refer to Including Additional Metadata.
Include document threads Ticked by default, to ensure all documents in the thread will be included in the export.
Untick this option if all documents in the thread are not required.
- The export will only include documents that have changed since the last export was run, such as documents that were created, modified, or updated.
- If this is the first time running the Export, you will need to generate the private key. Click Generate (or Regenerate Keys if an export was completed previously).
This generates a .pem file with private keys. - Click the link Your .pem file is ready. Click here to download it.
- Save the downloaded .pem file, to a secure location.
- Checkmark "I have downloaded the .pem file" to confirm you have done this.
- Move to Step 3 - Run the Export.
After setting the criteria for the documents that will be exported, you can continue to run the export:
- From the Export settings window, Click Next.
Note: If you have not applied any filters, a message will be displayed. Click Yes to continue, or No to return to the Export Settings to select from the Filters. - The Folder Structure will display. This determines the hierarchy that will be created by the Export. The structure list determines the top-level folder and any sub-folders for the Export.
Check this is correct for the export you want to run. You can make any changes to apply to the export as required. You can remove any metadata groupings from the Structure List, and add them from the Available list. You can also change the sequence of the metadata in the Structure List:
- Click to select what you want to move.
- Drag and drop it either to the Structure List to include it, or drag it to the Available List to remove it from the Structure List.
- Drag it up or down in the Structure List to change the sequence.
Note: Changing the structure after the export has completed will not update documents already exported.
- Click Next.
- A Summary of the export will be displayed, including an Estimated Document Count (including archived documents).
Note: Where practices have a large number of files to be exported, the Estimated Document Count may time out, but the user will be able to proceed with the export. - Click Create to request the Export.
- Move to Step 4 - Confirm the Bulk Export is Complete.
Note: The export must be finished before you move to the next step. Any changes to Document or Client Lists after this point will not be included in the Export.
After creating an export, the user will be redirected to Practice Settings - Documents - Export tab. The list of Exports will be displayed. The Export list displays all exports and works like any list in FYI, except users cannot search and changes cannot be saved.
Tip: An entry will also be added to Practice Activity when the export is requested. Users can click the Practice Activity entry to view the export details.
Once an export has been requested, it will run until it is finished. Each export shows a status and progress indicator:
Pending: The export has been requested. The Progress column indicates the percentage completion of the export.
In Progress: The export is running, and a grey progress bar is displayed.
Complete: The export has finished, and a green progress bar is displayed.
Note: For any long filenames, the filename of an exported file will be truncated to 100 characters.
To cancel an export while it is Pending or In Progress, select the export and click Cancel.
Using Cyberduck to Access and Extract Files to SFTP
Important Note: The export must be finished before starting these steps.
Downloading and installing the SFTP Client is a one-off process. We recommend using Cyberduck and the examples below are shown using this as the SFTP to access the exported documents.
Click the link below to download and install Cyberduck.
https://cyberduck.io/download/
When you start the Cyberduck application, the following screen displays.
The Cyberduck setup is now complete and you are ready to start the connection process.
- Click Open Connection.
- Choose SFTP (SSH File Transfer Protocol from the drop-down.
- Enter the Server as per your region below:
Australia and New Zealand:
s-87445f28b4fb451ea.server.transfer.ap-southeast-2.amazonaws.com
United Kingdom:
s-411b4a77b93640ae8.server.transfer.eu-west-2.amazonaws.com
Port - Do not change the Port number.
- Enter the remaining details as follows:
Username - Enter the full email that you use to log in to FYI. An email alias cannot be used to log into Cyberduck. This can be checked in the User Profile drawer. Refer to Managing Users.
Password - Leave the Password field blank.
SSH Private Key - Click Choose and select the .pem file downloaded from FYI.
No changes are required under 'More Options'.
- Click Connect.
- If an Unknown Fingerprint pop-displays, click Allow.
- When you log into the SFTP, you see a folder for the export with the folder name set as the date and time when the export request was started. Below this are the folders for the export structure that you specified.
The next step is to extract the files from Cyberduck.
Important Note: You must ensure you have adequate disk space and stable network connectivity to perform these steps.
If you want to calculate the size of the export in AWS before extracting:
- In Cyberduck, right-click over the folder you want to extract from.
- Click Info.
The following menu displays. - Click Calculate and let it run through all files to calculate the size of the export.
This may take some time, depending on how many files are hosted.
To download the files:
It is recommended to download the files to your local hard drive.
We do not recommend selecting OneDrive as the download location due to the syncing of documents during the download. Files can be moved to OneDrive after the download is complete, or copy them to a portable USB or hard drive as required.
To download the files:
- Right-click over the folder you want to extract from and select Download To.
Tip: To select multiple folders hold the Ctrl button and click on each of the required folders.
- In the Browse For Folder window, select the folder you want to save the files to.
- Click OK. The files will be downloaded to that location.
After the documents have been downloaded, the data remains accessible with Cyberduck, so the files can be extracted again if required.
Document Export CSV
After running the export, two reports are added to the export-data folder of the export:
- report.csv file: Contains details of each file included in the export and contains metadata of these files.
Tip: Expand the following to see the columns included in the report.csv file.
Document Name
File Path
Modified FileName (the filename as exported)
Document Type
Category
Reference Number
Created On
Created By
Last Modified On
Last Modified By
Client Name
Client Code
Export Code
FYI Client ID
Cabinet
Year category
Filing Status
Keywords
To
From
CC
Attachment
-
errors.csv file: Contains details of each file that was unable to be exported.
Note: The number of documents that were unable to be exported will be displayed in the Export Results within FYI. For example, Draft emails will not be able to be exported.
To locate any of the documents from the csv file in FYI, enter the Reference number from the spreadsheet into the Document List search field. Refer to Searching for Documents.
Running the Export Again
For any subsequent runs of the Export:
- You do not need to generate a private key again. The previously generated private key gives you access to the newly exported documents.
- If you need to change the Export structure, this can be done when running another export.
- If a new export is started, the files will be saved to a new folder with the folder name of the date and time when the export request was started.
- If you re-export the same documents this will not overwrite any previous exports.
To Run an Export again, follow the steps Running the Export.
Accessing the STFP as a Different User
The .pem file is linked to the email address of the user that requested the export.
If the STFP needs to be accessed by a different user, note the following:
If another user needs to temporarily access the SFTP:
- That person should contact the original user who ran the export and work with them directly.
If another user needs permanent access to the SFTP:
- A new private key will need to be regenerated. After regenerating, the new user will have access to previous exports as well as any new exports. The original user will not have access to previous or new exports.
Practices migrated from one FYI Practice to another FYI Practice
If documents have been exported using Bulk Export AWS and need to be re-imported to another FYI platform, they can be imported with the FYI Migrate App by selecting the Windows Explorer option.
When re-importing files to a new FYI account, note the following:
- Ensure that all report.csv files are included in the folder location 'export-data' for the Migrate App. This will allow this metadata to be applied to the imported documents.
- Where an email contains an Attachment, this column will display a Yes value. When performing the re-import a paperclip icon will be displayed against the email in the list view in FYI. Related attachments can be accessed by exporting the migrated email. Refer to Exporting Documents.
- FYI Document hyperlinks may break on import to the new practice, if the exporting practice no longer has access to the original FYI platform.
- When running a bulk export and configuring the Export settings to include Document threads, these will be filed as separate documents when re-importing to a new FYI platform.
Including Additional Metadata
If you want to retain additional metadata (such as Partner, Manager, or Tags) as part of the export, you can do so by exporting a document list separately from FYI:
Use the Documents list in FYI to export metadata.
Ensure the FYI Reference Number is included as the first column in the export.
Include any additional metadata columns you wish to retain (e.g., Partner, Manager, Tags).
When running the Bulk Export, select Include FYI Reference to append the reference number to each exported file name.
This allows you to cross-reference the exported documents with your metadata list using the FYI Reference Number.
Note: You can export up to 10,000 rows per list, so multiple exports may be required for large datasets.