Theo Todman's Web Page - Notes Pages


Status Reports

Status: Web-Tools (2018 - December)

(Text as at 06/01/2019 23:36:58)

(For earlier versions of this Note, see the table at the end)


Rationale for this Project


Summary of Progress during October - December 2018
  1. I spent 79.5 hours in 4Q18 on this Project, or related work (79.5 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2018). That's 51% of the planned effort (51% YTD). Overall, 13.3% of my Project effort in the Quarter was directed towards this project (making 13.3% YTD) - as against 22.2% planned (22.2% YTD).
  2. Again, I made no progress at all on the courses put out by Harvard
    "Harvard University - CS50 Introduction to Computer Science", and
    "Harvard University - CS50W Web Programming with Python and JavaScript".
    I still intend to make a serious attempt to complete the former sometime (I will need to re-start). However, I decided to precede it with the latter which is more immediately relevant.
  3. Early in the quarter I’d been on course to approximate budget, but decisions to meet or exceed budget on other projects – in particular my Thesis, Music and Chess – and two weeks’ illness – got in the way in the latter half of the quarter, when I did virtually nothing on this project, so I only achieved about 50% of plan in the end.
  4. However, I did manage to spend 55 hours developing my own site and did a lot of useful work.
  5. Completed items included:-
    1. Own Website:
      • Architecture
        1. As revealed by Spider: Old links in unregenerated pages. 8 Items.
      • Documenter
        1. Outstanding Developments by Category appearing all in Bold, and links to lists failing.
      • Notes
        1. Create Priority Task List Report12.
        2. As revealed by Spider: Links from Blog in Level 1 or 2 Printed Notes 51213 and 98114 failing. 141 items.
        3. As revealed by Spider: Links to sundry 'static' pages failing.
          1. 13 items fixed.
          2. 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
        4. As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
          1. Website: Maintenance Dashboard15. 14 items.
          2. Personal Identity Status Report16. 11 items.
          3. Summary Status Report17. 4 items.
          4. Actual Detail Summary Report18. 6 items.
          5. Chess Status Report19. 12 items.
          6. Etc. 3 items.
        5. As revealed by Spider: Consciousness Status Report20 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
        6. As revealed by Spider: Sundry links failed for Supervisions. 30 items.
      • Process
        1. Added 'time to regenerate' sub-totals & grand-totals to Website Maintenance Dashboard21.
      • Spider
        1. Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
      • Status
        1. Links occasionally missing from Summary Task List22 and Summary Task List (YTD)23.
      • WebRefs
        1. Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
    2. Other Websites:
      • Enigma
        1. Added 'EE' shortcut icon on all Webpage browser tabs
      • Sophie
        1. Added 'SB' shortcut icon on all Webpage browser tabs
  6. Full details for 18Q4 are given below:-
Website (Total Hours = 75)
  1. Website - Development (Total Hours = 55)
    • Create Timeline software: Add photos of Tom & Coxes Farm (10.5 hours)
    • Investigate Spider_Copy: Full_Link_Up_Levels_Gen. (2 hours)
    • Website - Generator - Add timestamps to MsgBox completion messages (0.25 hours)
    • Website - Generator - Add totals to Maintenance Dashboard (5 hours)
    • Website - Generator - Development Log - Bug Fixes (0.5 hours)
    • Website - Generator - Fixes re Broken Links revealed by Spider (15.5 hours)
    • Website - Generator - Investigating pgn4web Chess-game viewer (1.5 hours)
    • Website - Generator - Links from Blog in Level 1 or 2 Printed Notes 512 and 981 failing (1 hour)
    • Website - Generator - Links occasionally missing from Summary Task Lists (1.75 hours)
    • Website - Generator - Maintain consolidated Development Log (1 hour)
    • Website - Generator - Priority Task List Report (1 hour)
    • Website - Generator - Various fixes re Supervisions Notes Directory errors (4.75 hours)
    • Website - Generator - Web Spider - Check out & correct missing links (2 hours)
    • Website - Generator - Web Spider - Delete Raw_Links associated with Pruned Files (0.75 hours)
    • Website - Generator - WebRefs - Added Display text to WebRefs (2.25 hours)
    • Website - Generator - WebRefs - Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon (5.25 hours)
      → See "Software Development - Website - Development" (55 hours)
  2. Website - Education (Total Hours = 2)
  3. Website - Infrastructure (Total Hours = 5.5)
    • iCloud for Windows Installation (0.75 hours)
    • Julie's Laptop Support - Including Windows 10 Bugs & Upgrades (0.25 hours)
    • Microsoft Windows 10 / MS Office - Releases, Bugs & Periodic Re-boots (2.5 hours)
    • PC Backups / OneDrive (1.5 hours)
    • XMas Newsletter - photos & formatting (0.5 hours)
      → See "Software Development - Website - Development" (5.5 hours)
  4. Website - Maintenance (Total Hours = 12.5)
    • 18Q3 Status Reports (1 hour)
    • Renew Kaspersky License (0.75 hours)
    • Website - Correct WHOIS Data for theotodman.com Domain Name (0.75 hours)
    • Website - Generator - General: Tidy up Site - delete un-updated pages (1 hour)
    • Website - Generator - WebRefs - Manual URL Checks (0.5 hours)
    • Website - Periodic Full Regeneration (6.25 hours)
    • Website - Run Web Spider (1.25 hours)
    • Website - ZoomSearch database refresh (1 hour)
      → See "Software Development - Website - Development" (11.5 hours)
Website Others (Total Hours = 4.5)
  1. Website Others - Enigma Ensemble (Total Hours = 1.5)
  2. Website Others - Sophie Botros (Total Hours = 3)

Plans for the Near Future

I’ve had to halve the planned weekly effort on this project to 6 hours in order to allow for other commitments. While there’s still a lot on my development “priority list”, my hope for the current academic year was to focus on updating my technical competence. However, I can’t see how to fit it in. The Plan below is taken automatically from the Priority 1 items on my Development Log, as published in my Outstanding Developments24 Report.
  1. Own Website: Priority 1 Items By Category:-
    • Architecture
      1. Complete XRef-re-engineering project:-
        1. Ensure all links and link-pages use the new XRef table, and pension off the old tables.
        2. Check all link-types still work and fix any errors.
        3. Complete the auto-triggering of regeneration of “associated” link pages.
        4. Fix update bug in Convert_Webrefs.
        5. Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
      2. Review effectiveness of hyperlinking method in the light of PhD and Philosophy of Religion experience.
      3. Where possible, use ID rather than NAME for in-page hyperlinks
    • Authors
      1. As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages. 72 items.
      2. As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items.
    • Backups
      1. Review architecture to improve performance; Need to document first
    • Books/Papers
      1. Investigate whether multiple Subject/Topic/Subtopic usage leads anywhere (ie. are just the first (of 3) actually used). Fix anything amiss.
    • Documenter
      1. Provide Functional Documentation for Website Generator (using Notes)
    • Education
      1. Continue with Harvard CS50's Web Programming with Python and JavaScript: either complete it, or check it'll still run in 2019.
      2. Investigate Bootstrap.
      3. Plan what to do with "Sitepoint (Learnable) - Sitepoint Learnable Web Development Courses" and the eBooks in my possession.
      4. Read "PC Pro - Computing in the Real World".
      5. Read "Barnes (Russell), Ed. - Web Designer".
      6. Re-start "Harvard University - CS50 Introduction to Computer Science". Maybe just "audit".
      7. Understand Bitcoin & Blockchain better
    • Infrastructure
      1. Re-install iCloud: solve 'The upload folder for iCloud Photos is missing' problem
    • Notes
      1. Add "Note Alternates" to Note pages.
      2. Allow the option to concatenate Notes in the Printed version (ie. linearly embed them essay-style), rather than treating the hyperlinks as footnotes – but still keep the hyperlink & cross-referencing in place.
        1. For use as "disclaimers" - eg. for "Plug Notes".
        2. For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
      3. As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
      4. As revealed by Spider: Blog25 Achived Notes - sundry links to Printable Notes. Live Note OK. 110 items. 100 items outstanding.
      5. As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
      6. Investigate Note_Links: Section references seem to be incorrect
      7. Printable Notes: fix the bug whereby the “private” flag is round the wrong way.
      8. Suppress the publication of the Printable versions of Temp Notes
    • Photos
      1. Create Timeline software: Add photos of Tom & Coxes Farm
      2. Develop software & procedure to make adding more content to the photos pages easier to undertake.
    • Process
      1. As revealed by Spider: Sundry uncategorised. Refs failing. 17 items.
      2. Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
      3. Full Website Regeneration is now taking 27 hours. Investigate why so, and improve performance!
    • Spider
      1. Analyse the results of the data collection exercise and design a plan of campaign to fix broken Internal links and prevent recurrence.
        1. Correct the code so the problems discovered by the Spider don’t recur.
        2. Delete 'orphan pages' that are never linked to, ie. Use the Spider to prune redundant pages26 automatically where possible.
        3. Fix the historical data where errors are uncovered by the Spider. An easier task now the site has a full-regen function.
      2. The Spider was generating WebRefs. Procedurally, this ought not to have been possible.
        1. The major problem turned out to be because unprocessed27 URLs got added to the end of the last WebLinks_Tester_Brief page, which then got Spidered. I've stopped this happening, so hopefully the problem will not recur. The fix was made in 18Q2.
        2. However, 4 other creations appeared - dated 18/05/18 - from the run of 10/07/18. The creation date was from the previous spider run, but the IDs show that they were produced in the latest run.
        I've re-opened the case!
    • Technology
      1. Look into Sistrix Smart28. Errors and warnings itemised are:-
        1. Duplicate content: seems to be theotodman.com and Link (Defunct)
        2. Title Tags: Empty, too long, identical
        3. Page Not Found
        4. Filesize in excess of 1Mb
        5. Meta-Description: Empty
        6. Few words on Page
        7. H1: Not used, used multiple times per page, identical across pages
        8. Pictures: Alt attribute missing
    • WebRefs
      1. As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 116 items.
      2. As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 16 items.
      3. Documentation & Bug-fixes: Phase 2
        1. Re-document the procedures in the light of recent changes.
        2. Resolve issues generated / revealed by the spider.
        3. Investigate - and fix where possible - broken links.
      4. Reformat WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm
        1. Clarify 'truncated': Display, not link
        2. Allow more space for 'link returned', 'issue' and 'display text'
        3. The 'As Above" lines waste space. Only for Notes Archive? Consolidate onto single second line.
      5. Reformat WebLinks_Tester_Brief: Allow more space for 'link returned', 'issue' and 'display text'
  2. Other Websites: Priority 1 Items By Category:-

Summary of Progress to Date

This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
  1. Summary of Progress to Date29.
  2. Outstanding Developments30,
  3. Functional Documentation31,
  4. A summary of time expended across the years developing my website32 is at "Software Development - Website - Development".



In-Page Footnotes:

Footnote 4: Footnote 5: Footnote 7: Footnote 9: Footnote 10: Footnote 11: Footnote 26: Footnote 27: Footnote 28: Footnote 32:


Printable Version:



Table of the Previous 12 Versions of this Note: (of 71)

Date Length Title
10/10/2018 16:43:41 22078 Status: Web-Tools (2018 - September)
06/07/2018 18:56:10 16773 Status: Web-Tools (2018 - June)
05/04/2018 10:48:00 16588 Status: Web-Tools (2018 - March)
05/01/2018 00:11:31 13295 Status: Web-Tools (2017 - December)
09/10/2017 23:25:26 11848 Status: Web-Tools (2017 - September)
20/07/2017 14:34:05 11297 Status: Web-Tools (2017 - June)
10/04/2017 23:38:24 11470 Status: Web-Tools (2017 - March)
14/01/2017 20:18:14 13578 Status: Web-Tools (2016 - December)
14/10/2016 22:14:53 14567 Status: Web-Tools (2016 - September)
21/07/2016 16:12:00 13206 Status: Web-Tools (2016 - June)
05/04/2016 23:19:41 15377 Status: Web-Tools (2016 - March)
07/01/2016 00:59:02 15186 Status: Web-Tools (2015 - December)



Note last updated Reference for this Topic Parent Topic
06/01/2019 23:36:58 520 (Status: Web-Tools (2018 - December)) Status: Summary (2018 - December)

Summary of Note Links from this Page

Internet Technology and Philosophy Research - Internet Technology Research - Proposal Status: Actual Detail Summary (2007 - 2019) Status: Chess (2018 - December)
Status: Consciousness Studies (2018 - December + Ongoing) Status: Personal Identity (2018 - December) Status: Priority Task List (2019: January) Status: Summary (2018 - December) Status: Summary Task List (2019: January)
Status: Summary Task List (YTD: 18Q4 - 19Q1) Theo Todman's Blog Theo Todman's Websites Maintained Website - Outstanding Developments Website - Progress to Date
Website Generator Documentation - Control Page Website: Maintenance Dashboard      

To access information, click on one of the links in the table above.




Summary of Note Links to this Page

Internet Technology and Philosophy King's Maths Questions Simon - T1S1T1 Status: Consciousness Studies (2018 - December + Ongoing) Status: Priority Task List (2019: January)
Status: Summary (2018 - December), 2 Status: Summary Task List (2019: January) Status: Summary Task List (YTD: 18Q4 - 19Q1) Theo Todman's Philosophy Page Theo Todman's Website Maintainance History
Website - Progress to Date, 2, 3, 4, 5        

To access information, click on one of the links in the table above.




Authors, Books & Papers Citing this Note

Author Title Medium Extra Links Read?
Software Development Website - Development Paper Low Quality Abstract    



Text Colour Conventions

  1. Black: Printable Text by me; © Theo Todman, 2019
  2. Blue: Text by me; © Theo Todman, 2019




© Theo Todman, June 2007 - Jan 2019.Please address any comments on this page to theo@theotodman.com.File output:
Website Maintenance Dashboard
Return to Top of this PageReturn to Theo Todman's Philosophy PageReturn to Theo Todman's Home Page