Cloud Sync in C#

Windows APIs

• Cloud Sync Engines
  • Windows.Storage.Provider
    • Registers Sync Root (and some shell extensions)
    • More registry info
  • Cloud Filter API
    • Used in sync provider implementation

Windows C++ from C#

• .csproj Target OS
• extern keyword
• P/Invoke
• COM Interop
• Vanara

Other options?

• Source generation
• CsWin32

Samples/References

• windows-classic-samples (c++)
  • A sample using a local directory as the "cloud"
  • Isn't actually a full implementation: does not move updates between "client" and "server" directories
• C# with Vanara
  • a C# version of the windows-classic-samples CloudMirror using Vanara.PInvoke
  • Doesn't include IStorageProviderStatusUISource
• IT HIT UserFileSystem
  • A product that abstracts Cloud Sync Engine
  • Helpful documentation about some of the Cloud Sync concepts
  • UserFileSystem sample

Architecture

• BackgroundService
  • Sets client directory as search-indexable
  • Registers class objects for COM
  • Registers sync root at client directory
  • Connects to sync root events (e.g. fetch, rename, delete)
  • Starts watch for changes in client directory (to hydrate/dehydrate on attribute change, and to update remote)
  • Starts watch for changes in remote directory (to update client)
  • Wait for SIGTERM (ctrl+c)
  • Dispose/unregister watchers, sync root events, sync root, COM class objects
• Updating remote: IRemoteReadWriteService
  • Local implementation: LocalRemoteReadWriteService
• Updating client: PlaceholderService

Helpers

• Disposable / Disposable<T>
  • Do action on dispose without needing to implement a new IDisoposable class
• CancellationTokenAwaiter
  • Support for await cancellationToken;

Shell Extensions and MSIX

• Cloud Files API supports further shell extensions such as custom states, context menu commands, thumbnail provider, quota (preview)
• Shouldn't use managed code (C#)?
  • 2013: Do not use managed code
  • Microsoft recommends against writing managed in-process extensions
    • Not entirely sure what the distinction is between "in-process" and "out-of-process"
  • Seems to technically work, even without Single-Threaded Apartment (STA) and CoInitializeEx
    • Except haven't been able to get ThumbnailProvider to work
  • Can register with DI-generated factories!
• Implementing an interface: ComVisible, Guid
• CoRegisterClassObject and ClassFactory
• Package your app as an MSIX
  • Package.appxmanifest
    • CloudFiles extensions
    • "null" Clsid 20000000-0000-0000-0000-000000000001
    • Held back for a long time: apparently having empty DesktopIconOverlayHandlers broke it
    • IStorageProviderStatusUISource is in preview
  • Should use package project as startup project for local debug
• More shell notes

TODO

[x] Allow SyncRoot PopulationPolicy Full (on-demand placeholder creation) [x] Anticipate file conflict. Is it ok to resolve with newer LastWrittenTime? etag (hash) would be better, but would need to maintain state [] Queue changes until closed [] Convert to https://github.com/microsoft/CsWin32? [] Manager and windows service? [] "Manager" UI app to view, add, remove sync roots and backing services [] Persisted data? [] Windows service [] Shell? How to integrate with MSIX registration - does there need to be only one shell registration for all sync roots? [] Restart crashed sync roots? [] Each sync root as separate service?

SyncProvider / ClientWatcher

[] Cancel FetchData/FetchPlaceholders [x] Create events DTO for ConnectSyncRoot [x] New client file -> cloud storage provider [x] Convert to placeholder [x] Mark as in-sync [x] Don't dehydrate client [x] Update - FileSystemWatcher Changed is very chatty, and no way to really check what changed on the file Compare LastWritten to server? [x] Rename [x] Delete [] Combine SyncProvider and ClientWatcher?

Local Server

[x] New server file -> New placeholder [x] Don't hydrate client [x] Update server file -> ? [x] Rehydrate downloaded file [x] Update Pinned file without unpinning it [x] Update offline file (for some reason causing a Fetch that throws a 380 error) [] Update downloaded file when can't obtain lock (clear in sync, separate interval process to attempt sync) [x] Rename server file -> ? [x] Delete server file -> ? [] Changes (create, update, rename, delete) while offline -> ?

Syncing

• Uploading large files client->remote tends to break. [] Pause remote watcher while uploading? [] Uploading status to hold queue of uploading files?

Add/Remove sync root through app

[] Start [] Sync changes since last time service ran (offline, stopped) [] Uninstall [] Unregister sync roots