Sponsored Content
« Tuesday Overnight Open Thread (4/5/22) | Main | The Morning Report - 4/6/22 »
April 06, 2022

Daily Tech News 6 April 2022

Top Story

  • Waiting for house news.

    Update: There's a buyer for this house as well so I've upped their offer by $25k. I'll be cross if this one gets away because I've already set up a shopping list at Ikea.

    Update: And now I'm probably going to be cross. The other buyers lost their house in the floods, and they've already paid the deposit, so unless they can't exchange contracts tomorrow the owners are going to take their offer.

    Also, apologies are due to unknown real-estate agent #1 who I thought was playing games. That property - which had been on the market for quite a while - has sold.

  • Had a report that one of my (virtual) servers was port scanning someone. I'm not sure if it was legit, and a scan of the server showed nothing untoward, but I have to take it seriously.

    First step was to block unwanted outbound traffic at the firewall. The first thing I do with any server is to block unwanted inbound traffic, but everyone* does that these days so all the major exploits sneak unwanted data in on trusted connection. - like the huge Log4j mess a few weeks ago.

    Easy peasy because if you screw up the firewall settings on a virtual server you can get in via the virtual console. I've done this on physical servers on the other side of the planet, with complex networking arrangements and no console, and that is nerve-wracking.

    Next step was to rebuild that server entirely since all it runs is a proxy server (Caddy) and it's needed an update for a while. Before doing that I checked on the backups to make sure if anything went wrong I could easily restore and, well, f*ck.

    The backup drive on the backup server is not responding at all. The syslog is full of ZFS deadman events. And the load average is a personal record, and I've worked on some pretty big servers.

    That is not a happy 10-core bunny.

    The backups run A->B->C where A is the active server and C is a an archive server that can collect lots of daily backups and B is broken. Which means that for, uh, ten weeks, no backups were going off site from that origin server to anywhere.

    So, I arranged three off-site backups and then rebuilt the proxy server with the latest software and the new firewall rules and basically tweaked the config file at random and kept restarting it until it all worked. I don't know why that was necessary, but it was. At least I could flick the routes back and forth internally and didn't need to wait for DNS to update.

    Then back to the backup server. Look at the I/O stats. It's written how much data? 18 petabytes? No wonder it's not working, the SSD must be fried.

    Wait... Insert commas manually. 18 terabytes. That's not much at all.

    Kill the stuck processes (all 24,000 of them). Load average goes up. Kill the parent process... Load goes up even more.

    Guess it's reboot time.

    And... It sits there with ping working but no other sign of life for ten minutes. (This one doesn't have a remote console either. I grabbed it during the datacenter fire last year and took what I could get.)

    It does have a button to remotely power cycle the server but the page asks you not to do that if the server can ping because they'd rather have a technician look at it before the evidence of whatever the problem was disappears.

    So I start writing a tech support ticket and just as I'm about to submit it the server comes back up and is working perfectly as if there was never anything wrong.


    How was your evening?

    * Not everyone.

  • Twitter is adding an edit button, maybe. (The Verge)

    "Protecting the integrity of that public conversation" is of course Newspeak for eradicating dissent.

Tech News

Disclaimer: No, we are not having fun yet.
digg this
posted by Pixy Misa at 04:20 AM

| Access Comments

Recent Comments
San Franpsycho: "Good evening morons and thanks wd for all the meme ..."

Dr. T: "This post seems contemptuous toward the brave men ..."

BignJames: "Might I suggest that, one night the ONT should fea ..."

Archimedes: "Not only was the Titanic a dinghy compared to the ..."

zombie Kapitänleutnant Walther Schwieger: "My, such fat, juicy targets! ..."

Walter Freeman: "[I]@12: "... only comment section more Maoist than ..."

jewells45 fuck cancer: "Hi nurse! Hanging in there! ..."

nurse ratched : "Jewells! Hi there sweetheart! How are you? ..."

Tonypete: "Might I suggest that, one night the ONT should fea ..."

Taq, Rickrolled by Jesus: "Thanks WD. Imma read the content but first... ..."

mindful webworker - reading between the lines: "… Was trying to come up with a greeting, ..."

jewells45 fuck cancer: "Good evening! ..."

Recent Entries

Polls! Polls! Polls!
Frequently Asked Questions
The (Almost) Complete Paul Anka Integrity Kick
Top Top Tens
Greatest Hitjobs

The Ace of Spades HQ Sex-for-Money Skankathon
A D&D Guide to the Democratic Candidates
Margaret Cho: Just Not Funny
More Margaret Cho Abuse
Margaret Cho: Still Not Funny
Iraqi Prisoner Claims He Was Raped... By Woman
Wonkette Announces "Morning Zoo" Format
John Kerry's "Plan" Causes Surrender of Moqtada al-Sadr's Militia
World Muslim Leaders Apologize for Nick Berg's Beheading
Michael Moore Goes on Lunchtime Manhattan Death-Spree
Milestone: Oliver Willis Posts 400th "Fake News Article" Referencing Britney Spears
Liberal Economists Rue a "New Decade of Greed"
Artificial Insouciance: Maureen Dowd's Word Processor Revolts Against Her Numbing Imbecility
Intelligence Officials Eye Blogs for Tips
They Done Found Us Out, Cletus: Intrepid Internet Detective Figures Out Our Master Plan
Shock: Josh Marshall Almost Mentions Sarin Discovery in Iraq
Leather-Clad Biker Freaks Terrorize Australian Town
When Clinton Was President, Torture Was Cool
What Wonkette Means When She Explains What Tina Brown Means
Wonkette's Stand-Up Act
Wankette HQ Gay-Rumors Du Jour
Here's What's Bugging Me: Goose and Slider
My Own Micah Wright Style Confession of Dishonesty
Outraged "Conservatives" React to the FMA
An On-Line Impression of Dennis Miller Having Sex with a Kodiak Bear
The Story the Rightwing Media Refuses to Report!
Our Lunch with David "Glengarry Glen Ross" Mamet
The House of Love: Paul Krugman
A Michael Moore Mystery (TM)
The Dowd-O-Matic!
Liberal Consistency and Other Myths
Kepler's Laws of Liberal Media Bias
John Kerry-- The Splunge! Candidate
"Divisive" Politics & "Attacks on Patriotism" (very long)
The Donkey ("The Raven" parody)
Powered by
Movable Type 2.64