• H2GD Part 54: Windows 10 butterfly effect

    Here’s a sequence of events I wouldn’t wish on anyone. What started out as one website running low on disk space ended up with me having to reload some of my local databases from backups.

    1. Customer reported that they were having problems updating files
    2. I checked disk space and my admin emails and confirmed my suspicions… they were getting low on server space
    3. While I was there I looked at some of my other admin emails
    4. I decided now was as good a time as any to apply some plugin updates to one of my sites
    5. But the latest version of the oik base plugin didn’t install
    6. I checked the host; the .zip file was missing
    7. I tried to ftp it from my local server’s wp-contents/uploads/2016/02 folder
    8. But I could not access my own .zip file from the sub-directory
    9. I checked other local servers and found other new files which I wasn’t allowed to access, but I noted that WordPress generated thumbnails were OK.
    10. The problems seemed to date back to somewhere between the 13th and 16th Feb
    11. I was aware of the following changes:
      • I’d recently upgraded to Windows 10
      • I’d had a problem with Kaspersky Internet Security; this had required me to delete and reinstall it
      • I knew that a Windows update ( KB777778 ) had been applied
      • And also that my machine had recently rebooted itself without any notification… I was busy using it at the time!
    12. Since I’d not found anything on Google search, and believing no harm would be done by it, I initiated a system restore
    13. Then, while waiting patiently for the restore to do its business, I continued to search for the true cause of the problem, using my iPad
    14. I found it almost immediately
      http://www.howyoudo.info/index.php/how-to-fix-windows-server-upload-file-inherit-permissions-error
    15. Some time later the system restore happily announced to me that it had failed, displayed a message for a brief moment, and then restarted my machine
    16. I then set about changing the permissions on my C:\Windows\Temp directory
    17. And I also dealt with the inaccessible files in the uploads folders
    18. But when I came to test my local websites I got the following message
      Error establishing a database connection
    19. This happened on some of the sites
    20. I then found that the MySQL service was Stopped
    21. I started it and tried again. At least three sites were failing, the rest seemed OK
    22. Of course the 3 sites that were failing were the ones I’d been using most recently
    23. Investigation of the MySQL error log showed a whole slew of problems.
    24. To cut a long story short I dropped the databases then restored them from my most recent backups

    I hope this problem never happens again.

    TL;DR

    Extract of selected MySQL messages in the .err file

    Some of the messages logged in the MySQL error log are:

    2016-02-24 13:30:30 830  InnoDB: Operating system error number 2 in a file operation.
    InnoDB: Some operating system error numbers are described at
    InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
    2016-02-24 13:30:30 2680 [ERROR] InnoDB: Could not find a valid tablespace file for 'wporg/wp_options'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
    2016-02-24 13:30:30 2680 [ERROR] InnoDB: Tablespace open failed for '"wporg"."wp_options"', ignored.

    There were quite a few of the above; one per corrupt tablespace. Then there were a few of these.

    InnoDB: Error: tablespace id is 3155 in the data dictionary
    InnoDB: but in file .\lalucouk\wp_options.ibd it is 2076!
    2016-02-24 13:30:56 1478  InnoDB: Assertion failure in thread 5240 in file fil0fil.cc line 796
    InnoDB: We intentionally generate a memory trap.
    InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
    InnoDB: If you get repeated assertion failures or crashes, even
    InnoDB: immediately after the mysqld startup, there may be
    InnoDB: corruption in the InnoDB tablespace. Please refer to
    InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
    InnoDB: about forcing recovery.

    I think they’re on to something here! But if I submit a detailed bug report I expect the response will be “Well what did you expect?”.

    I also made the mistake of changing my configuration file without fully understanding the consequences. I believe this led to me getting these messages.

    2016-02-24 19:11:44 7608 [Note] InnoDB: Starting crash recovery.
    2016-02-24 19:11:44 7608 [Note] InnoDB: Reading tablespace information from the .ibd files...
    2016-02-24 19:11:45 7608 [Note] InnoDB: Restoring possible half-written data pages 
    2016-02-24 19:11:45 7608 [Note] InnoDB: from the doublewrite buffer...
    2016-02-24 19:11:47 1488  InnoDB: Error: table 'lalucouk/wp_options'
    InnoDB: in InnoDB data dictionary has tablespace id 3155,
    InnoDB: but a tablespace with that id does not exist. There is
    InnoDB: a tablespace of name lalucouk/wp_options and id 2076, though. Have
    InnoDB: you deleted or moved .ibd files?
    InnoDB: Please refer to
    InnoDB: http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html
    InnoDB: for how to resolve the issue.

    Current situation

    Having restored the corrupt databases and rebooted I’m left with one Warning message.

    2016-02-24 22:07:55 838 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

    Has something good come from all this?

  • I’ve backed up all the important databases
  • and deleted some redundant databases
  • I’ve rerun my performance test suite
  • The system’s almost working as well as it did under Windows 8.1
  • and now I’ve finally got around to rebuilding my local copy of WP-a2z.com with my latest improvements to oik-shortcode… which can’t be a bad thing.

,

Published:

Last updated:

February 26, 2016

Categories

Tide times from tidetimes.co.uk

Tide Times & Heights for Northney on
Saturday, 23 October 2021

Tide times from tidetimes.org.uk

Tide Times & Heights for Northney on
23rd October 2021
01:47 High Tide ( 4.6m )
06:56 Low Tide ( 0.61m )
14:04 High Tide ( 4.67m )
19:11 Low Tide ( 0.68m )