Scripts Gone Wild!

[jcsnider]

So I made a script that would run on my servers each night -- it was very simple.. it would run a list of specified commands (for backing up databases), then compress a list of folders into a single archive, password protect everything, and then upload the archive to Google Drive.  It even allowed me to specify how many backups to keep and delete the oldest when past that threshhold. Finally, I was emailed a log of how the backup went each night just so I could keep an eye on it.

 

This script worked for over a year but last week I noticed that I stopped receiving the emails -- and sure enough no new backups had been uploaded to google drive in over a week. I thought this was a little strange so I went to investigate.  Needless to say my script had a "On Failure: Retry" policy -- and the backups were failing, and they kept retrying and they never stopped. We had over 7 days of backups trying to upload at the same time over and over again for awhile...  It's fixed now.. but I'm glad our bandwidth limit is just a suggestion!

 

Anyone else have weird stories where scripts have gone off the reservation? Share them!