Using PowerShell to setup custom log ingestion gives a great flexibility to automate the whole process specially when you need to setup custom logs in bulk. To ingest logs (text files in .log format) from Windows and Linux machines to Azure Log Analytics workspace, you can create and configure Custom Logs as data source in Log Analytics. Log Analytics agent installed on a machine connected with Log Analytics polls for new log data every 5 mins. However, it may take an hour or more  for the initial data from a new custom log to appear in Azure Log Analytics Workspace.

When you create a custom log, Log Analytics will append it with _CL. Once data is on-boarded in Azure Monitor, you can parse the raw data into multiple columns and store the query text as a function which can be used as query to display logs.

Note: Adding custom log for mapped network drives is not yet supported as the network drive does not appear as a disk volume in a computer.

The following sample script creates and configures a workspace with custom log data source configuration.

$ResourceGroup = "az-rg-log-analytics"
$WorkspaceName = "az-LogAnalytics-01"
 
 $CustomLogs = @"
    {
    "customLogName": "CustomServerLog",
    "description": "Example custom log datasource",
    "inputs": [
        {
            "location": {
            "fileSystemLocations": {
                "windowsFileTypeLogPaths": [ "c:\\logs\\*.log" ]
                }
            },
        "recordDelimiter": {
            "regexDelimiter": {
                "pattern": "((\\d{2})|(\\d{4}))-([0-1]\\d)-(([0-3]\\d)|(\\d))T((\\d)|([0-1]\\d)|(2[0-4])):[0-5][0-9]:[0-5][0-9]",
                "matchIndex": 0,
                "matchIndexSpecified": true,
                "numberedGroup": null
                }
            }
        }
    ],
    "extractions": [
        {
            "extractionName": "TimeGenerated",
            "extractionType": "DateTime",
            "extractionProperties": {
                "dateTimeExtraction": {
                    "regex": null,
                    "joinStringRegex": null
                    }
                }
            }
        ]
    }
"@
New-AzOperationalInsightsCustomLogDataSource 
-ResourceGroupName $ResourceGroup 
-WorkspaceName $WorkspaceName 
-CustomLogRawJson "$CustomLogs" 
-Name "Server Log Collection"

These are the supported formats when a log delimiter is a timestamp:

Format	RegEx Format
YYYY-MM-DD HH:MM:SS	((\\d{2})|(\\d{4}))-([0-1]\\d)-(([0-3]\\d)|(\\d))\\s((\\d)|([0-1]\\d)|(2[0-4])):[0-5][0-9]:[0-5][0-9]
M/D/YYYY HH:MM:SS AM/PM	(([0-1]\\d)|[0-9])/(([0-3]\\d)|(\\d))/((\\d{2})|(\\d{4}))\\s((\\d)|([0-1]\\d)|(2[0-4])):[0-5][0-9]:[0-5][0-9]\\s(AM|PM|am|pm)
dd/MMM/yyyy HH:MM:SS	(([0-2][1-9]|[3][0-1])\\/(Jan|Feb|Mar|May|Apr|Jul|Jun|Aug|Oct|Sep|Nov|Dec|jan|feb|mar|may|apr|jul|jun|aug|oct|sep|nov|dec)\\/((19|20)[0-9][0-9]))\\s((\\d)|([0-1]\\d)|(2[0-4])):[0-5][0-9]:[0-5][0-9])
MMM dd yyyy HH:MM:SS	(((?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Sept|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)).*?((?:(?:[0-2]?\\d{1})|(?:[3][01]{1})))(?![\\d]).*?((?:(?:[1]{1}\\d{1}\\d{1}\\d{1})|(?:[2]{1}\\d{3})))(?![\\d]).*?((?:(?:[0-1][0-9])|(?:[2][0-3])|(?:[0-9])):(?:[0-5][0-9])(?::[0-5][0-9])?(?:\\s?(?:am|AM|pm|PM))?))
yyMMdd HH:mm:ss	([0-9]{2}([0][1-9]|[1][0-2])([0-2][0-9]|[3][0-1])\\s\\s?([0-1]?[0-9]|[2][0-3]):[0-5][0-9]:[0-5][0-9])
ddMMyy HH:mm:ss	(([0-2][0-9]|[3][0-1])([0][1-9]|[1][0-2])[0-9]{2}\\s\\s?([0-1]?[0-9]|[2][0-3]):[0-5][0-9]:[0-5][0-9])
MMM d HH:mm:ss	(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\\s\\s?([0]?[1-9]|[1-2][0-9]|[3][0-1])\\s([0-1]?[0-9]|[2][0-3]):([0-5][0-9]):([0-5][0-9])
MMM d HH:mm:ss two spaces after MMM	(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\\s\\s([0]?[1-9]|[1-2][0-9]|[3][0-1])\\s([0][0-9]|[1][0-2]):([0-5][0-9]):([0-5][0-9])
MMM d HH:mm:ss	(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\\s([0]?[1-9]|[1-2][0-9]|[3][0-1])\\s([0][0-9]|[1][0-2]):([0-5][0-9]):([0-5][0-9])
dd/MMM/yyyy:HH:mm:ss +zzzz where + is + or a – where zzzz time offset	(([0-2][1-9]|[3][0-1])\\/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\\/((19|20)[0-9][0-9]):([0][0-9]|[1][0-2]):([0-5][0-9]):([0-5][0-9])\\s[\\+|\\-][0-9]{4})
yyyy-MM-ddTHH:mm:ss The T is a literal letter T	((\\d{2})|(\\d{4}))-([0-1]\\d)-(([0-3]\\d)|(\\d))T((\\d)|([0-1]\\d)|(2[0-4])):[0-5][0-9]:[0-5][0-9]