network outage

Network has been down since about 10:30 this morning. There is evidently "an equipment issue affecting our local area", according to customer service. ETR is 10pm tonight (PDT).

And.. that's all I know for now

yay, more windstorms

FYI: Forecasters are predicting serious wind storms for tonight in the Seattle area. We are fine for the moment, but if we go down, that'll be why.

Seattle Area Wind Storm

Actually, it's really more of a big ass wind storm sitting off the entire West Coast of ths US (short version: You don't want to be in a small craft in the water anywhere between Mendocino and British Columbia right about now). What is extra-special fun is this one involves unusual winds from the east that are more likely to confuse the trees and make them fall over.

And I'm now seeing the sequence of brief-but-getting-longer power flickers and sounds of pole transformers blowing up that usually converges to Out For A While. Just a heads up.

NOAA says things are going to remain At Least Annoying until around 4am.

Update (10pm):  Looks like we finally got nailed. 300K customers without power so this will indeed probably be a while...

Update (1am, Wed):  "As of 1 a.m. we have 474,032 customers out of power. We anticipate a multiple day outage...". Well, then...

Update (7pm, Wed):  Latest ETR for our area is 8pm, 11/23,
...as in Saturday (!!)

Update (4pm, Fri):  Outage map now says they're down to 118,540 customers without power. But that still includes us. At least one neighborhood within a half-mile now has a crew assigned, so that's something.

Update (7:30pm, Fri):  Looks like we are back up. It's a bit confusing since, from the logs, it looks like the machine rebooted at around 3:15pm but we evidently didn't get the network back until about 15 minutes ago. Anyway, enjoy.

phone line work

So they're working on our phone line.

Evidently there's a short of some kind in some cable out in the neighborhood that's been preventing dial tone from happening (that we haven't noticed because we mostly don't use the land-line anymore) that's evidently not been affecting the internet service (and right now they're doing experiments and internet still seems to be up [guy says, "It's on a different frequency"]) -- and hopefully it'll continue to stay up. But here's a heads up just in case something stupid happens.

Update (6:30pm):  And done. We have dial tone. And the network never actually got disconnected, so yay for that.

Power Failure

Power's been out for about 15 minutes now. There's been heavy rain the past few days and it looks like they're dealing with rather a lot tonight, so I'm guessing this one may take somewhat longer to come back. We'll see.

Update (1am):  They're now giving me an ETA of 2:30am.

Update (6am):  There are now 380 outages affecting 25,000 people in the Seattle area. Last text message status update said, "Work is underway" (no further details), but PSE website's outage map is saying ours is still in the "Outage reported" phase (no cause located, no crew assigned). Looks like this is going to be a while.

Update (6:55am):  "Assessing damage." They've apparently gotten as far as figuring out that "Trees/vegetation" had something to do with it. Progress!

Update (8:20am):  "Crew assigned" + activity observed at a plausible location while driving W to school. Now down to 15,900 customers without power.

Update (9:10am):  ...and we are back. Yay.

server panic

So... this was not your imagination:  We did, in fact, have a server panic on Saturday around 3:30pm, and I didn't actually notice until Tuesday. Suffice it to say, I need to set better alarms for this sort of thing. I also evidently let myself get distracted between bringing it back up and making the obligatory blog post. Not my best week.

Anyway, something stupid happened in a verb_args() call, and that's as much as I know at this point. I've turned up the checkpointing frequency to once per hour (it was at 24 hours and we lost about 23 hours of activity).

For future reference, if you're noticing LM down and you're not seeing anything about it here and it's been more than an hour (+ usual allowances for sleep if it's middle-of-the-night PST), you should feel free to DM me on Mastodon (@wrog@mastodon.murkworks.net) or Failbook -- sad to say, these channels seem to be more reliable than email these days.

Network Outage

We went off the air at around maybe 11:30am. Automated phone tech support thingie says they're working on it and it should be resolved by 10am tomorrow (8/9/2022).

And... that's all I know at this point. Bleah.

Update (2pm Mon.):
Now they're saying tomorrow (Tuesday) at 5pm. ++Bleah.

Update (7:30pm Mon.):
Oh, weird. It looks like things came back five hours ago (2:27pm if I believe what connected_seconds() is telling me for various people) and they didn't notify me the way they said they would. Which might mean it's going be bouncing a bit. But for now, things seem to be up again.