Theo Todman's Web Page - Notes Pages
Status Reports
Status: Web-Tools (2024 - December)
(Text as at 01/02/2025 00:28:48)
*** THIS IS NOT THE LATEST VERSION OF THIS NOTE ***
(For the live version and other versions of this Note, see the tables at the end)
Rationale for this Project
- This Project was alluded to briefly in a footnote on research methodology in my original Research Proposal1 under the head Research - Internet Technology2. When last at Birkbeck, I wrote a more extensive paper3 defending the Project and describing its rationale. I continue to take this Project further. There’s still a lot to do: still quite a few items on the “wish list”. It is fairly critical as an enabler for my research, so I need to get a move on as I had wanted it all out of the way before I re-started4 formal research.
- For documentation on my website follow the links below:-
- Functional5 Documentation6.
- Technical7 Documentation.
- Other Websites8
- Active
- Billericay Mayflower Bridge Club: since November 2021, on and off
- Old House Friday Bridge Club: a website that I’d created (amongst the other ‘small sites’) back in 2013. Revitalised in 2023.
- Enigma Ensemble: I created and maintained a small website for a music group Julie and I attend and which I now run. I lapsed the bespoke URL in favour of this sub-site of my own website but now intend to reinstate it. I’ve updated the site in the light of the retirement of our Director and to reference our new location.
- a1waspsandbees: Commercial site for a bridge-partner. Minimal involvement beyond retiring duplicate sites.
- Inactive
- Sometime around 2005, I created a website for Dr. Sophie Botros, one of my supervisors at Birkbeck, but we then lost touch and it got maintained (very badly) by some desktop support outfit. In 15Q2 I took it back on again and spruced it up a bit, and maintain it periodically until 19Q3, when it was taken on by a professional outfit, Bookswarm. The “Sophie Botros” link in this bullet is now to their version of the site. My site looked like this.
- I created and / or ran a multitude of other bridge websites, but as of January 2018 I have either handed them over or mothballed them9:-
- I established the Hutton Bridge Club (Defunct) Website in 11Q4 using the standard Bridgewebs service, but with a couple of competitions using my own routines. This was handed over in 15Q3. I briefly took it on again in March 2020. It needed a spring clean, but I was waiting to see whether the club (and its members) survived the coronavirus pandemic, and I handed it back on that account. The club went ‘on-line only’ but has now closed for lack of support.
- In 15Q1, I took over the support and development of the Essex Contract Bridge Association (ECBA) website, which also uses Bridgewebs, but is very much larger. I wrote a lot of code10 to make this job less tedious. The site was handed over in 17Q4.
- For several years, I collected data11 on bridge activity in the Billericay/Brentwood area (initially needed for a project to set up a new consolidated club) by “scraping” data off web pages, consolidating it into a database and modelling it in various ways.
- I used this data to generate websites with a multitude of ladders for small clubs (8 small Essex Bridge websites). These are now mothballed, with the exception of the Old House one noted above.
- I created and maintained a new website for the First Class Bridge Academy, giving it “small clubs” ladders (8 further “small sites” for Bernie) as these were easy to maintain with little intervention.
- As of March 2020 I took over the Mountnessing Bridge Club (Defunct) website. Earlier, in 16Q3, I created the Mountnessing Bridge Club Archive website, using the vast bulk of the pages from their legacy site, as the club had moved to Bridgewebs and lost its historical data. Sadly, the Club has now been wound up, but I intend to keep the historical record.
- I created a website12 for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles: Acts: Test Site.
Summary of Progress during October - December 2024
- I spent 113.5 hours in 24Q4 on this Project, or related work (113.5 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2024). That's 58% of the planned effort (58% YTD). Overall, 15.2% of my Project effort in the Quarter was directed towards this project (making 15.2% YTD) - as against 24.2% planned (24.2% YTD).
- As can be seen, having more than doubled my weekly time allocation to 15 hours for this Quarter, in order to catch up on important outstanding developments, I managed only to spend less than 60% of this time allocation and mostly not on what I’d planned.
- The ‘spoilers’ were a host of infrastructural issues (which don’t need rehearsing here) and taking over a bridge-friend’s commercial website. But the major time-stealer was work on the LRB Archive covered by my General Culture13 Project.
- Completed items included:-
- Own Website:
- Backups
- Purchase & commission new 5Tb WD Backup Drive. Copy 2Tb backup drive contents across.
- Books/Papers
- Created Journals table & Updated functor_21 for Thesis - Journals14
- Created LRB table & Updated functor_21 for "LRB - London Review of Books"
- Bridge
- Correct processing for Partner's NGS calculation
- Infrastructure
- Migrate 2 x Laptop + Desktop from Norton AV to Microsoft Defender: Norton non-renews 13/11/24
- Notes
- Active Note Groups with Notes Updated in the last 7, 14, 28 days: producing incorrect results. Important as there are links from my Home Page. Added Temp_Note_Timestamp column to Notes table.
- Other Websites:
- Full details for 24Q4 are given below15:-
Website (Total Hours = 100)
- Website - Bridge Development (Total Hours = 2.75)
- Website - Bridge Maintenance (Total Hours = 5.25)
- Website - Development (Total Hours = 41)
- Website - Education
- Website - Infrastructure (Total Hours = 40.25)
- Backup Drive: Investigate, Purchase & install 5Tb WD drive (1.25 hours)
- EE - New Contract - Julie's Phone (1 hour)
- EE - New Contract + Hub (Main House): Purchase & Installation (12.25 hours)
- Email Mailbox Pruning: Also queries to NamesCo (7 hours)
- Fix Bibleworks Keyboard (0.25 hours)
- Helping Ann with Emails and Zoom (1.5 hours)
- Investigating and installing Bitwarden (Free Edition) (0.75 hours)
- iPhone Ringtones (0.25 hours)
- Julie's Blackberry - Sundry troubleshooting (0.5 hours)
- Julie's Laptop - Sundry troubleshooting (0.5 hours)
- Microsoft Edge - Trojan removal (0.5 hours)
- Microsoft Windows & MS Office - Releases, Bugs & Periodic Re-boots (0.75 hours)
- Namesco Renewals (0.25 hours)
- New Outlook: IMAP not syncing (5.25 hours)
- PC Backups / OneDrive (0.25 hours)
- Researching & Purchasing corded mouse (1 hour)
- Tommy's Party Invites (1 hour)
- Toshiba Laptop - Occasional Windows Update (0.5 hours)
- Uninstalling Norton Antivirus and enabling Windows Defender (1.25 hours)
- XMas Mug - photos & formatting (0.5 hours)
- XMas Newsletter - editing, photos, formatting & printing (3.75 hours)
→ See "Admin - Website - Admin & Maintenance" (40.25 hours)
- Website - Maintenance (Total Hours = 3.25)
Website Others (Total Hours = 13.5)
- Website Others - A1WaspsAndBees
- Website Others - Mayflower DBC Maintenance
- Website Others - Small Bridge Sites
Plans for the Near Future
- Since I didn’t do what was intended last quarter, I’ve retained the weekly time allocation at 14 hours because otherwise there just isn’t enough time to accommodate all the things I want to do. I hope this push will be enough to make progress on important items.
- The Plan below is taken automatically from the Priority 1 items on my Development Log – as published in my Outstanding Developments25 Report.
- Own Website: Priority 1 Items By Category:-
- Architecture
- Complete XRef-re-engineering project:-
- Ensure all links and link-pages use the new XRef table, and pension off the old tables where appropriate.
- Check all link-types still work and fix any errors.
- Complete the auto-triggering of regeneration of “associated” link pages.
- Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
- Complete Documenting26 the process!
- Backups
- Determine why flag on directories "not to be backed up" gets reset
- Document and enable regular running of the de-duplication function ('pruning') of the backup disk. Keep a full record of deleted duplicates.
- Enable Backups to be undertaken on both my PC & Laptop and write reconciliation software.
- Incorporate new 5Tb WD Backup Drive into backup system.
- Investigate historic error messages
- Investigate Record-count discrepancies:-
- How do website files work as far as counts are concerned?
- Why aren't they recorded in Backup_History, nor the fact that the website was backed up?
- Different counts depending on whether new or old laptop is backed up. Investigate 63k discrepancy - lower on new laptop.
- Review architecture to improve performance; Need to document27 first
- Sort out the contents of my C:Drives, OneDrive and Emails as documented in my Backup Documentation
- Books/Papers
- For Books and Papers - add link pages - analogous to:-
→ BooksToNotes_
→ BookCitings_
→ PapersToNotes_
→ PaperCitings_
linked to from:-
→ BookSummaries_
→ PaperSummary_
and (maybe) from:-
→ BookPaperAbstracts_
→ Abstract_
But, the links are FROM the Book / Paper, rather than TO
For Books, ensure that all the links from all the Book's Papers are also included
- Import pdfs downloaded from "LRB - London Review of Books" into databases.
- Sub-Topic ID has reached 996: What happens after 999? Research and Fix
- Bridge
- Bridge Results Export: Correct residual problems following code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug.
- After run, while the database closes OK, MS Access remains active and has to be crashed via Task Manager.
- If the Export is run a second time, as is required for Partner's calculated NGS to show correctly, without closing the database first, we get 'permision denied' when trying to create Bridge_Results_Theo.htm.
- Small Systems: Correct residual problems following code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug.
- After importing results via 'Joint Project Conversion', while the database closes OK, MS Access remains active and has to be crashed via Task Manager.
- After running the Mini-Wedsite Generator, while the database closes OK, MS Access remains active as a ghost Background Process and has to be crashed via Task Manager. Until this is done, MS Access won't open.
- Longer term - correct the fudge that simply re-creates Small Systems for the current year (hard coded to 2024).
- Documentation
- Website Generator Documentation - Audio28: Review & Complete Documentation
- Website Generator Documentation - Author Narratives29: Complete Documentation
- Website Generator Documentation - Citations30: Complete Documentation
- Website Generator Documentation - Code Documenter31: Complete Documentation
- Website Generator Documentation - Create Notes Web Pages32: Complete Documentation
- Website Generator Documentation - Cross-Referencing33: Complete Documentation
- Website Generator Documentation - Functional Overview34: Review & Complete Documentation
- Website Generator Documentation - Language / Ling Pages35: Complete Documentation
- Website Generator Documentation - Links & Link-Pages36: Complete Documentation
- Website Generator Documentation - Monthly Reporting37: Complete Documentation
- Website Generator Documentation - New Items38: Complete Documentation
- Website Generator Documentation - Note References & Reading List39: Complete Documentation
- Website Generator Documentation - PID Note, Book & Paper Usage40: Complete Documentation
- Website Generator Documentation - Precis Export41: Complete Documentation
- Website Generator Documentation - Prune Website42: Complete Documentation
- Website Generator Documentation - Spider & System Backups43: Complete Documentation
- Website Generator Documentation - Thesis Dashboard44: Complete Documentation
- Website Generator Documentation - Thesis Technology45: Complete Documentation
- Website Generator Documentation - Time Recording46: Complete Documentation
- Website Generator Documentation - Timelines (Photos & Blog)47: Complete Documentation
- Education
- Read "PC Pro - Computing in the Real World".
- Read "White (Ron) & Downs (Timothy Edward) - How Computers Work: The Evolution of Technology".
- Infrastructure
- Fatal Database Errors: ‘Cannot open any more databases: Run-time error 3048’.
- When I then try to compact and repair the database - the first port of call when there's a strange problem - it complains that it is locked by the Administrator. Then the database won’t close and has to be zapped by Task Manager. Only then can I delete the lock file, but this doesn't resolve anything as the problem recurs. I think the 'too many databases' error message is misleading.
- This started on Tuesday 27th August 2024 with Bridge_Web_Generator.accdb on Desktop. I stopped running it thereon and it was fine for a day on my Laptop, then the problem recurred.
- Looking on-line, this seems to be a known Microsoft bug for which a fix is awaited. It looks like the garbage dispoal routines (after set db = nothing) aren't working. It's not just where you're sloppy and don't do this and rely on the system to do garbage collection: I did changes to 'Small Systems' processing, but it made no difference.
- My main 'Thesis' database is working fine - I managed a full Regeneration starting late on 31st August. It may be that the bug was fixed under Windows 11 but not Windows 10. However, I'm not taking the risk of running my Bridge database on my Desktop in case it mucks up my Thesis database. The failure of Compact / Repair of my Bridge database is particularly worrying, but my Thesis databse C/Rs fine on my Laptop (and Desktop). This is a relief, as this database becomes unusable after a few days otherwise as it reaches the 2Gb limit
- Currently I'm waiting and seeing. Re-importing all the elements into a new Bridge database made no difference.
- However, so that I can continue running my Bridge processing, I've raised a couple of Development Items to enable
→ Small Systems
→ Bridge Statistics
- Migrate from Chrome to Bitwarden for password management
- Notes
- Allow the option to embed blocks of re-usable text in Notes (ie. linearly embed them essay-style) rather than using hyperlinking. Slightly analogous to Note_Snippets or Sub_Notes, but more flexible.
- For use as "disclaimers".
- The identical block of text in the Dashboard48 to that in the Note Thesis - Method & Form49 that describes the Thesis.
- For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
- As a more controllable alternative to 'Printable Notes'.
- Correct processing for 'References & Reading List' section in Thesis - References50. There are over 7,000 but only 7 items appear!
- Determine why all my Status Reports from September 2021 are missing from Notes_Archive. Seek to recover them from backups.
- Develop method for (manually) deleting otiose Archived Notes: ie. ensure links work for remaining Notes!
- Enhance PID Note, Book & Paper Usage51 to include a section listing works cited in my Thesis but not read
- Investigate Note_Links: Section references seem to be incorrect
- Notes_Links: Segment & Provide Jump Table (Eg. For Thought52)
- PID Thesis References - De-duplicate / suppress and / or change sort order / aggregation for non-anthology book chapters
- Update Personal Page53 & Home Page
- Upgrade Auto_Reference_Notes to reference Sub-Notes: Used to only affect one note - Somerset Maugham - but now may be urgent
- When using the '#' reference to Notes, allow the omission of the Note ID for links within the same Note.
- Papers
- Split PDFs Available on Local Website54 page into alphabetical sub-pages
- Photos
- Timeline software55: Add photos for Bakers' Coxes Farm Development Phase III
- Timeline software56: Add photos for Henry's Biography
- Process
- Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
- Spider
- Investigate reduction in size of Web_Generator_Performance database that holds Raw_Links.
- Suppress MsgBox for 'Near to 2Gb' database size check (just use Debug).
- One-off deletion of rather useless links: EMail, TheosStyle.css, TT_ICO.png, #Top, ...
- Remove code that generates these links, and check them.
- Investigate removal of website 'root' (C:\Theo's Files\Websites\Theo's Website\) from Raw_Link and Full_Link. This would save 200Mb, but needs careful checking - especially against the Directory_Structure and related tables and code.
- If the above 'root' change is to go ahead - one-off updates + code changes required.
- Use Edge rather than Internet Explorer for External Links Spider (use Bing AI)
- Status
- Reorganise Status Reporting:-
- Merge Consciousness Project57 with Thesis58 (as a sub-project).
→ Merge historic plan & actuals
→ Adjust current Thesis plan, actuals & task list.
- Create a new Project: Culture & Politics
→ Move 'Thesis background reading / writing and 'Aeon' to this new project.
→ Create project plan and move historic actuals to it.
→ Adjust 'Thesis' historic actuals and task list: move relevant items to new project.
- Make sure the Status reports - both manual and automatic - still work
- WebRefs
- Create a page showing the use of WebRefs more clearly. The pages accessed via Theo Todman's Web-Links Test Webpages are of little use other than as an audit. Also, there are many broken links not picked up by the Spider. Need pages that use meaningful names.
- Documentation & Bug-fixes: Phase 2
- Re-document the procedures in the light of recent changes.
- Resolve issues generated / revealed by the spider.
- Investigate - and fix where possible - broken links.
- Find a way of recording Missing / Unknown Webrefs (eg. Webrefs 8395 and 8342 in Archived Notes 1292 and 1317) other than debug.print: create table, then suppress message for known problems
- Investigate items flagged as defunct. Populate Defunct_Explanation in WebRefs_Table. Consider use of FairUse (Link (Fair Use)) for documents no longer available that I'd downloaded.
- Investigate WebRefs with Issue = 'URL Translated OK': does the translation really work? How?
- Visitor Stats - cmdVisitorStats_Click (and following) - loops and requires 'DAO.DBEngine.SetOption dbmaxlocksperfile,1000000' to be typed in the Immediate window. Re-engineer.
- Other Websites: Priority 1 Items By Category:-
Summary of Progress to Date
This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
- Summary of Progress to Date59.
- Outstanding Developments60,
- Functional Documentation61,
- A summary of time expended across the years developing my website62 is at "Software Development - Website - Development".
In-Page Footnotes
Footnote 4:
- Well, in a sense, I’ve missed the boat as I’m now putting effort into my research, though in an informal basis, so will need to continue with both projects in parallel.
- This was always likely to be necessary, as new features will always arise in use. It’s a prototype methodology, after all.
Footnote 5:
- This is very tedious to produce and consequently is both incomplete and out of date.
Footnote 7:
- This is much more fun, as it’s a purely technical task.
- I’ve written a vastly-improved general-purpose technical documenter for MS Access.
Footnote 9:
- It’s a shame to abandon the “mini websites” with all their ladders, as it’s rather well done.
- However, I couldn’t waste time on these after I’d abandoned bridge.
Footnote 10:
- In particular, for the ECBA “Victor Ludorum” competition.
- I cannot hand any of this code over, so the tedium will return, though not to me!
Footnote 11:
- I had agreed to share this data sometime early in 2018
- But will wait until asked again, as I doubt it’ll be of any real use to anyone.
Footnote 12:
- It used to exist in two versions, live and test.
- Pete decided not to renew the license for the live site, now it has achieved its purpose, so only the test site remains.
Footnote 15:
- Note that where fixes or small enhancements are made to a previously “completed” development, I don’t announce it again against the list of “completed” items above, though the work appears in the full list for the quarter.
Footnote 62:
- As distinct from developing other peoples’ websites – time which is also recorded against this project, but not against this task.
Live Version of this Archived Note
Table of 12 Earlier Versions of this Note (of 94)
Table of the 4 Later Versions of this Note
| This version updated |
Reference for this Topic |
Parent Topic |
| 01/02/2025 00:28:48 |
520 (Status: Web-Tools (2024 - December)) |
Status: Summary (2026 - March) |
Summary of Notes Links from this Page
To access information, click on one of the links in the table above (if any).
Summary of Note Links to this Page
To access information, click on one of the links in the table above (if any).
Text Colour Conventions
- Blue: Text by me; © Theo Todman, 2026