Prerequisites

1. You need to install the FS Crawler before you can use it, please follow Filesystem Crawler Installation and Administration guide.
2. Files and Folders that need to be Crawled need to be visible for the CW Data Collector for the standard Docker implementation this meens they need to be mounted at:

/var/dockershare/containershare/XXX

Where XXX represents one or many folders and files.

      3. Remote Folders from Windows or Linux machines can be mounted  by following Remote file systems guide.

Reading time: 5 minutes

Creating Crawler

Navigate to Crawlers tab in the Cogniware Data Collector administration (accessible at https://192.168.112.212/datacollector).

Click Add Crawler button.

Fill in the Name of the Crawler, you may also add Description.

For the Crawler configuration, you need to provide the following parameters:

• Connector Class: Select the Connector Class you have installed in Filesystem Crawler Installation and Administration.
  
• Connectors: Select the FileSystem connector you want to use. To configure the Connector please follow Filesystem Crawler Installation and Administration - for each FileSystem you need new configuration.
  
• Object Types: Select files.
• inputPath: Path to folder that should be crawled. All subfolders are automatically crawled too. (Path to folder that need to be crawled has to be mounted on docker container which is running CWDC. Folder where /var/dockershare/containershare is mounted in docker container is /containershare).
• extensionsInclude: (Optional) Limit files that will be crawled to only defined extension. If none provided, all files will be crawled. Wildcards are not supported. Do not include dot.
• extensionsExclude: (Optional) Limit files that will be crawled to exclude defined extensions. If none provided, none will be excluded. Wildcards are not supported. Do not include dot.
• Data Handler: Select Data Handler you created in Filesystem Crawler Installation and Administration.

Click the Add Crawler button.

Congratulations! Crawler configuration is done. We can start crawling!

Please refer to Working with Crawlers on how to start, stop or change the Crawler.