daemon.json file overwritten once the worker node joines to the cluster

[asreedharan]

I'm using GKE on-prem 1.9 version.  I want to keep insecure registries entry  in daemon.json file and I made those entries in the Windows template that I'm using for user cluster creation. But after building the user cluster, I could see all the entries in daemon.json file had been overwritten by some default values. How can I keep some customized entries in daemon.json file in Windows worker nodes.

[abdelfettah]

This is something you should avoid doing.
Is your registry insecure (http) or does it use a self-signed certificate ?

[asreedharan]

its insecure one

[abdelfettah]

That's typically not something we would recommand you to do. And GKE will not allow you to change the insecure config of Docker in a easy maner. If you restart the nodes that change will be reverted. 

What you can do is run a DeamonSet on the cluster to change the settings in the deamon.json each time a new node is create and/or restarted.

[asreedharan]

its a very good thought,  could you please share if you have a sample daemon set manifest that can be used for Windows worker nodes.

[abdelfettah]

I have published an article a while back to do something similar. It's for Linux and covers both Docker and Containerd runtime. But my solution is for using registries with Self-Signed certs. Maybe you can find some inspiration there https://medium.com/p/b37b5fd1f982. The code the Daemonsets is in github https://github.com/boredabdel/gke-private-registries