But... but... that'll wean citizens off the tits of Big Energy! And that'll not do for Washington & State officials that feed off Corporate influence, ahem, "donations."

The worst DevOps mistake I ever made:

Assignment: On ~1,000 -physical- RHEL systems, change the default run level from command line to GUI (don't ask).

Solution: Hey, all our config files are controlled by Puppet, so this'll be easy!

(If you don't know what Puppet does, it enforces file configurations, so if you change a single file on the Puppetmaster, that change is pushed out to all servers running Puppet)

Ok, all I need to do it edit a single file, change a single number in said file and issue a single command: reboot. Easy-fuckin'-peasy.

The file I need to change is /etc/inittab -- this file tells a Linux system which "run level" it should initiate upon booting up. runlevel 3 is command line and runlevel 5 is a GUI like Gnome or some other tragic perversion of the whole reason you run Linux in the first place. All I had to do was change from runlevel 3 to runlevel 5. And reboot.

So simple; so stupidly simple.

So stupidly simple at 3:00am. When I hadn't slept all night. On a production network. When I'm working from home away from the office. On a Saturday when no one is in said office.

I make my change and save it, then push it to the version control system. Puppet picks it up and pushes the change to ~1,000 physical computers.

Done and done!

Remember I mentioned that I had to change a single file AND execute a single command: reboot?

Here's where things go tragically wrong.

My changes worked PERFECTLY. Everything did exactly what I told it to: Puppet changed the file, and rebooted the servers.

Only they keep rebooting. They keep rebooting over and over and over and over. I can't access any server on the network. Worse, while I'm trying to figure out WTF I did wrong, the 30 minute time-out I'd set on our alerting system, Nagios, expires.

Did I mention that I pushed this change to ~1,000 servers? ~1,000 servers that won't stop rebooting and aren't reporting into Nagios, thus being marked as down?

At 3:31am, on Saturday morning, the pages to ALL the on-call engineers began. One page per engineer per machine. About one every two seconds. And I'm getting paged, too -- except some of the pages are Nagios and some are utterly irate engineers who want to know exactly WTF is going on and I can't tell which is which because I'm getting text-spammed like crazy.

And those servers? They just keep right on rebooting.

At that point, I felt the kind of existential dread that only people who work in IT know -- the kind of dread that arises a picosecond after you've hit ENTER and realized you've type 'rm -rf /' or some-such -- because I knew at that very second exactly what I'd done wrong.

I'd typo'd "5" and made it "6" in the runlevel. And pushed it to ~1,000 -physical- servers. And then rebooted them ALL.

"So," you're asking, "Whyfor is runlevel 6 a big deal?"

Because of this:

runlevel 3: command line.
runlevel 5: GUI
runlevel 6: REBOOT THE FUCKING COMPUTER.

What I'd done was told every production server on our network to reboot as soon as it rebooted, which leads to another reboot, which leads to another reboot, lather rinse repeat.

At 3:45am on Saturday morning, I knew that every person in IT would have to drive into the office, visit every production server with a bootable USB key, change the BIOS to boot off the key, boot the server into Single User Mode, change the damned file by hand, then reboot the server. This takes about 10 minutes per server -- times ~1,000.

I learned a number of valuable lessons that day:

1. DOUBLE CHECK YOUR FUCKING WORK.
2. See lesson #1
addendum: filing for unemployment insurance in Washington state is amazingly easy.

And that was the very last time I ever worked on physical hardware. To this day, if it's not in the cloud, I ain't fucking touching it.

Here endth the lesson.

Great news...the Japanese Giant Hornet has recently established itself in Washington state.

Giant hornets are bee predators. They can decimate a hive in hours. Japanese honey bees have a defense against them...they can vibrate their bodies generating enough heat to kill the hornets but not enough to kill themselves. Italian and other European bee species don't have that defense. If giant hornets spread, they could be a death nail in the already struggling American bee/honey industries.
Also worth noting, they kill up to 50 people a year in Japan, but far more in China. They could be worse than killer bees in multiple ways. For instance, they can sting through a bee suit, each one can sting multiple times, and they can thrive in cold that killer bees can't survive.

*quality masochism

The rate of spread (Ro) is said to be between 2.5 and 4.

That is the same as saying X to power of 2.5 or 4.


Would be best to shut down any County that is infected. Nip it in the bud.

The infected guy in NC last week got it from Washington state old folks home, Now we have7 official cases and nest week we should be 20+

If we can slow down the rate of spread then the Hospitals can deal with this more effectively. I think this wont happen and facilities will be over burden is short order.


On the bright side this burn through the world in few months.
Hope we all are stand standing then.

Your video, The Robots are coming for Washington State Apples, has made it into the Top 15 New Videos listing. Congratulations on your achievement. For your contribution you have been awarded 1 Power Point.

Washington state apples? Um....that's New Zealand, not Washington.

Not sure how this could work on established trees unless they're espaliered.

2 more comments have been lost in the ether at this killed duplicate.

This video has been seconded as a duplicate; transferring votes to the original video and killing this dupe - dupeof seconded with isdupe by eric3579.

Double post *dupeof=https://videosift.com/video/The-Robots-are-coming-for-Washington-State-Apples

This video has been nominated as a duplicate of this video by newtboy. If this nomination is seconded with *isdupe, the video will be killed and its votes transferred to the original.

*quality circular phenomenon! This was on the local news recently about a spot on the Snoqualmie River ... a rare occurrence in Washington state:

The times they are a'changin' [FINALLY!] Bobo and ... as @Khufu already pointed out ... Bill was 'fucking impeached'. Get with the program you crazy/sad clown. If you read my comment about what the Washington State GOP Chair thinks is the reasoning behind Trumps indiscretion, listen for yourself:



Match made in ... [fill in the blank]?

Oh and ... Bill's not running for president.

So since Hillary won big in Washington when all voters were allowed to vote, but we go by caucuses here....

Shouldn't Bernie, to be fair, cede those delegates to her? Because Democracy? Because Voice of the People?

http://www.thestranger.com/slog/2016/05/24/24121754/hillary-clinton-donald-trump-lead-in-pointless-washington-state-primary

http://www.thestranger.com/slog/2016/05/24/24121759/clinton-winning-washington-state-democratic-primary

My county went overwhelmingly for Sanders both in the caucus and the primary. Bunch of old progressive hippies. They make me proud.

*doublepromote

I'm hoping that the angry Sanders folks who admire Mr Oliver will watch this and understand that this year is just politics as usual, and has nothing to do with the specific players.

If you are in a swing state, please don't let your anger keep you from the polls on election day. Please don't help Trump become president.

I live in Washington State. A solidly blue state. My vote doesn't matter.

If you are in a swing state, yours does.

I wish I had her faith in the delegates switching their vote, but when 70% of votes go to Bernie (Washington state) yet NONE of the state's super delegates (all elected officials mind you) cast for Bernie, you have to wonder if they really are a safeguard against despotism or simply a means of protecting a party's status quo.