Show Posts
46
Open Discussion / The search for a near perfect FSW.
« on: August 03, 2018, 08:25:38 PM »
FileSystemWatcher (FSW) is pretty useful to monitor a certain folder for new file creation, a text file that usually contains some result data. The file is processed by first parsing its content, next send its parsed data on to a SQL DB and finally move the file to an archived folder.
The key here is to know when process of file creation is complete. Most solutions are either relying on regular interval timer check or use several of FSW notify filters ( CreationTime, FileName, LastWrite, Size ) to trigger event and then trying to open the file to check for its completion status. This approach is fine for low rate of file creation, but if there's a lot of files created in a short period of time, buffer overflow and missing file watching can occur.
FSW use a limited & non-paged memory and therefore long file name, deep level of sub directories, filter can all affect its performance. Increase the InternalBufferSize can help but MS only recommend its maximum size of 64KB.
Perhaps, the most logical choice is the LastWrite notify filter. Oddly enough, LastWrite will actually trigger events twice, it is the second trigger mark the end of file writing and so it's good time to open the file to parse its content.
The next logical one is the Size NotifyFilter, it trigger only once and appear to be at the end of file writing. I added the while loop for self assurance, may not needed:
The key here is to know when process of file creation is complete. Most solutions are either relying on regular interval timer check or use several of FSW notify filters ( CreationTime, FileName, LastWrite, Size ) to trigger event and then trying to open the file to check for its completion status. This approach is fine for low rate of file creation, but if there's a lot of files created in a short period of time, buffer overflow and missing file watching can occur.
FSW use a limited & non-paged memory and therefore long file name, deep level of sub directories, filter can all affect its performance. Increase the InternalBufferSize can help but MS only recommend its maximum size of 64KB.
Code: [Select]
Private Sub FileWatcher()
Dim fsw As New FileSystemWatcher With {
.Path = sourceDir,
.InternalBufferSize = 65536,
.IncludeSubdirectories = True,
.Filter = "*.DAT",
.NotifyFilter = (NotifyFilters.LastWrite)
}
AddHandler fsw.Changed, New FileSystemEventHandler(AddressOf OnChanged)
fsw.EnableRaisingEvents = True
End Sub
Perhaps, the most logical choice is the LastWrite notify filter. Oddly enough, LastWrite will actually trigger events twice, it is the second trigger mark the end of file writing and so it's good time to open the file to parse its content.
Code: [Select]
Private _fileCreated As Boolean = False
Private Sub OnChanged(ByVal source As Object, ByVal e As FileSystemEventArgs)
If _fileCreated Then 'catching the second trigger event
ParseFileContent(e.FullPath)
'End If
_fileCreated = Not _fileCreated
End Sub
The next logical one is the Size NotifyFilter, it trigger only once and appear to be at the end of file writing. I added the while loop for self assurance, may not needed:
Code: [Select]
Private Sub OnChanged(ByVal source As Object, ByVal e As FileSystemEventArgs)
Dim fileSize As Long = 0
Dim currentFile As New FileInfo(e.FullPath)
While Not (FileIsReady(e.FullPath))
fileSize = currentFile.Length
Thread.Sleep(25)
currentFile.Refresh()
End While
ParseFileContent(e.FullPath)
End Sub
Private Function FileIsReady(ByVal path As String) As Boolean
Try
Using file = System.IO.File.Open(path, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
Return True
End Using
Catch __unusedIOException1__ As IOException
Return False
End Try
End Function
