feat(windows): Delphi CE interactive build workflow

[MattGyverLee]

Summary

Lets developers on the free Delphi Community Edition tier build the still-Delphi parts of the tree, even though CE 11+ deliberately blocks the dcc32 CLI that build.sh normally drives. None of the compiler / runtime stack moves; this is purely build-environment work plus docs.

CI is unchanged. KEYMAN_DELPHI_CE is unset by default, so build.sh keeps invoking msbuild.exe exactly as before.

Split off from #16039 per @mcdurdin's review — that PR bundled the CE workflow with the Delphi 11/12 source-compat patches. This PR is the CE half; the compat half is #16043.

Depends on #16043 for the source-compat patches and the KEYMAN_DELPHI_VERSION knob that lets the IDE find Delphi 11/12.

Why this matters for the Delphi removal roadmap

Delphi 10.3 Pro is no longer obtainable, and CE 10.3 has been removed from Embarcadero's downloads. New contributors who want to help finish #4599 currently hit a wall: they can install CE 12, but CE blocks dcc32 and build.sh falls over before reaching any Delphi project. This PR fixes the wall — without committing to keep Delphi alive.

What's in the change

1. CE-aware delphi_msbuild

resources/build/win/environment.inc.sh: delphi_msbuild() now checks KEYMAN_DELPHI_CE. When =1, it prints an IDE-build prompt naming the .dproj and waits for Enter instead of invoking msbuild.exe. The pre-build (rc.exe, manifest gen) and post-build (binary copy, codegen) steps in build.sh run normally around the prompt, so the developer only has to drive the Delphi compile by hand.

This is the approach @mcdurdin proposed in #16039 (comment) — gate by version in the existing shell script rather than parallel PowerShell helpers.

2. Docs

• docs/build/windows.md: one-line pointer to windows-delphi-ce.md from the existing "Delphi 10.3 CE no longer available" warning, so future CE users find the workaround instead of dead-ending.
• docs/build/windows-delphi-ce.md: ~940-line guide — prerequisites, registry library-path setup, source patches required on chore(windows): Delphi 11/12 source compatibility #16043, the canonical build order (including the tsysinfox64 -> .bin -> tsysinfo_x64.res chicken-and-egg), install-and-overlay workflow, debugging, troubleshooting catalog, and a section labeling which patches are safe to upstream. Written against Delphi 12 Athens CE specifically (currently the only free tier); Delphi 11 CE works with the same approach modulo a couple of paths.

3. engine.groupproj fix

windows/src/engine/engine.groupproj: drop the inst\insthelper reference. The path doesn't resolve — insthelper.dproj lives at windows/src/engine/insthelper, not .../inst/insthelper — and IDE "Build All" was failing on it. build.sh-based builds were never affected because they don't walk the groupproj.

Technically independent of the CE work, but bundled here because IDE "Build All" is the path CE users actually exercise. Happy to split into its own PR if reviewers prefer.

4. .gitignore

Patterns for the per-script log files that the CE workflow writes (configure.log, core-build.log, etc.).

Reviewer notes

• The PR's behaviour-changing surface is exactly one if [[ "${KEYMAN_DELPHI_CE:-}" == "1" ]] branch in delphi_msbuild(). Everything else is docs / .gitignore / the orthogonal engine.groupproj cleanup.
• The PowerShell helper scripts from the original chore(windows): support Delphi 11/12 compilation + Delphi 12 CE dev workflow #16039 are gone in this revision; the workflow runs entirely through KEYMAN_DELPHI_CE=1 ./build.sh.
• The doc file is named windows-delphi-ce.md (not windows-d12.md) so the same workflow guide applies if/when a future Delphi 11 CE user shows up; the body is Delphi 12-specific because that's what's been actually tested.

User Testing

User testing not required: build-environment / docs change with no runtime behaviour modification. CI will exercise the non-CE default path (KEYMAN_DELPHI_CE unset); the CE flow is manually validated via the steps in docs/build/windows-delphi-ce.md.

Relates-to

Relates-to: #4599
Depends-on: #16043
Replaces: #16039 (split into #16043 + this PR)

[keymanapp-test-bot]

User Test Results

Test specification and instructions

ERROR: user tests have not yet been defined

[keyman-server]

This pull request is from an external repo and will not automatically be built. The build must still be passed before it can be merged. Ask one of the team members to make a manual build of this PR.

[MattGyverLee]

Took a hard look at the groupproj-batching path. Sharing the analysis so we can pick the cheapest workable design.

Coverage and order — what each groupproj would buy us on CE

Groupproj	Delphi sub-projects in dir	In groupproj	Missing	Internal order
engine.groupproj	keyman, kmcomapi, tsysinfo, tsysinfox64, insthelper (5)	4	insthelper	WRONG: keyman;kmcomapi;tsysinfo;tsysinfox64 — tsysinfo is attempted before the tsysinfox64.exe it embeds exists
desktop.groupproj	kmshell, kmbrowserhost, setup, kmconfig, insthelp (5)	5	—	OK internally
developer.groupproj	TIKE, setup, kmconvert, kmdbrowserhost + ext BPLs	3	kmdbrowserhost + design-time BPLs	OK internally

So before any other consideration, adopting engine.groupproj would need a one-line reorder (tsysinfox64 before tsysinfo) plus restoring insthelper at its real path. That's a change to the non-CE flow we'd like to avoid here.

What build.sh actually does between Delphi compiles

The reason per-.dproj prompts fit cleanly is that the build chain interleaves a lot of work that the groupproj knows nothing about. Per child build.sh:

rc.exe (version.rc, manifest.rc, project-specific .rc)
   → manifest.in → manifest.xml generation
   → delphi_msbuild <project>.dproj    ← CE prompt currently fires here
   → sentrytool_delphiprep (symbol-upload prep, per-arch)
   → tds2dbg (TDS → PDB conversion)
   → copy outputs into install layout

Plus the cross-project hand-offs that aren't in any .dproj or .groupproj:

• (a) devtools → keyman/setup/kmshell. devtools.exe -buildmessageconstants emits MessageIdentifierConsts.pas (~1500 lines) from windows/src/desktop/kmshell/xml/strings.xml. devtools.exe -buildsetupstrings emits ~32 Keyman.Setup.System.Locale.<bcp47>.pas files from windows/src/desktop/setup/locale/*.xml. All .gitignored. Skipping this gives F1026 File not found: '...MessageIdentifierConsts.pas'.
• (b) build_standards_data → TIKE. Emits five BCP-47 registry .pas files in common/windows/delphi/standards/ (the largest is LangTagsRegistry.pas at ~2.4 MB). All .gitignored. Skipping gives F1026 File not found: '...BCP47SubtagRegistry.pas'.
• (c) tsysinfox64 → tsysinfo. tsysinfo.dproj (Win32) embeds the Win64 helper tsysinfox64.exe as tsysinfo_x64.res. After building tsysinfox64.dproj, build.sh copies bin/Win64/Debug/tsysinfox64.exe → tsysinfo/tsysinfox64.bin, then runs rc tsysinfo_x64.rc in windows/src/engine/tsysinfo/ to produce the .res, then builds tsysinfo.dproj. Skipping any step gives F1026 File not found: 'tsysinfo_x64.res'.
• (d) kmcmplib CLI → TIKE runtime. TIKE links to kmcmplib-19.dll, which is built CLI via ./developer/src/kmcmplib/build.sh build — no IDE involvement, but a clean CE build that skips it produces a TIKE that throws "kmcmplib-19.dll not found" on any compile action.
• (e) keyman.dproj → kmshell runtime. kmshell compiles without keyman.exe — failure is at runtime: kmshell shows "Keyman failed to start" when enabling a keyboard.
• (f) regsvr32 kmcomapi.dll → kmshell startup. windows/src/engine/build.sh install does this; without it, CoCreateInstance returns REGDB_E_CLASSNOTREG.

(c), (d), and the codegen halves of (a) and (b) are exactly the "post-Delphi hand-offs" build.sh interjects between Delphi compiles. The groupproj has no equivalent and can't fire rc.exe, devtools.exe, or shell copies.

What "use the groupproj" would actually require

To get one prompt per group, we'd have to restructure each child build.sh so that under CE:

1. All children's pre-build (rc.exe runs, manifest.in → .xml) fire first across the whole group.
2. ONE IDE prompt fires for the groupproj.
3. All children's post-build (sentrytool_delphiprep, tds2dbg, copies, codegen hand-off (c)) fire across the whole group.

That's a CE-only fork of how every child build.sh is structured — and the engine group still needs the tsysinfox64 → tsysinfo reorder (or it fails inside the IDE the same way) plus an insthelper add-back. Per-dproj prompts side-step both: the existing dependency order in build.sh already gets it right, including the (c) hand-off as a post-build step on tsysinfox64.

Proposal

Ship as-is. The PR-as-it-stands is a one-if-branch CE-only change in delphi_msbuild that fits between the existing rc.exe and sentrytool/tds2dbg calls in every child. ~30 prompts on a top-level clean build, but:

• no reorder of engine.groupproj (no non-CE flow change),
• no fork of any child build.sh (no non-CE flow change),
• chicken-and-egg (a)–(f) all keep working because the existing build.sh order is what handles them,
• design-time BPLs / insthelper / kmdbrowserhost don't need a special case — they're driven by their own build.sh and get the same per-dproj prompt.

Happy to revisit if there's a tighter design I'm missing, but under the "minimal CE-only changes" constraint, that's where I land. Also happy to add this analysis as a section to docs/build/windows-delphi-ce.md so the next CE contributor doesn't re-litigate it.

[MattGyverLee]

I had Claude plow through the build and tell me what to do every time I hit an error until I could compile and run everything. I hit all those A-F chicken/egg compile problems above trying to follow the build. I wonder if the CLI version would hit some of the same blockers if it was put onto a fresh new machine, but I can't test that (and don't want to start from scratch again while in "hackathon" mode. I only know it worked on my machine after all the intervening workarounds.

[mcdurdin]

There's a lot of redundant and incorrect information in the docs. I think this needs a thorough rework

[mcdurdin]

This doesn't seem right; TKeymanPaths.KeymanDesktopInstallPath() does not hard code this path as it is read from the registry.

To run keyman.exe properly so that it can interact with elevated UI processes (uiAccess=true in the manifest), it needs to be signed with a valid cert, and be installed in a hardened path such as Program Files. But for debugging, you can run keyman.exe unsigned with uiAccess=false, in non-elevated processes. All other components should be runnable outside of Program Files, although there are libraries that have to be located.

[MattGyverLee]

You're right — I hadn't actually followed TKeymanPaths.KeymanDesktopInstallPath() to see where the path comes from. It reads from a registry key the Keyman installer populates, not a hardcoded path. Reworded the §1 prereq (install simplifies setup by populating that key rather than being strictly required) and updated the §7 troubleshooting entry to attribute the failure to the missing registry key. URL corrected to keyman.com/windows.

[mcdurdin]

This is not right. keyman.com/windows. s/overlay/debugging. The reason for installing Keyman is to simplify the install environment but it is possible to debug without Keyman installed.

[MattGyverLee]

Reworded — install "simplifies setup" rather than "required", and it's possible to debug without an official install. URL corrected to keyman.com/windows. See the TKeymanPaths reply above for the underlying reason.

[MattGyverLee]

Once this PR is clean, I need to test with a full uninstall/rebuild to make sure. It seemed very much to be required to move forward.

[mcdurdin]

This is fragile because if we make changes to the project the docs will need to be updated so we don't get out of sync. I don't like having this kind of thing twice really.

[MattGyverLee]

Kept the explicit list because when I hit F1026 File not found: jvcl.inc on my first .dproj open, chasing it down took me longer than the recipe took to write — I'd rather leave a working recipe with drift risk than a pointer a future CE contributor has to reconstruct. Added a note that the first six paths mirror DELPHIINCLUDES (verified against the file) and the other eleven don't. Fair on the drift risk though, this is a doc that'll need refreshing if either side changes.

[MattGyverLee]

Reduced some of the fragility with a grep.

[mcdurdin]

these should not be necessary

[MattGyverLee]

Agreed, false positive from the delphi docs.

[MattGyverLee]

Removed — those were the source-compat error entries. §7 now points to #16043 for source-compat and only troubleshoots CE-specific things.

[mcdurdin]

Any movement on this @MattGyverLee? We're need to close stale pull requests so please let me know where you plan to take this.

[MattGyverLee]

@mcdurdin , looking at this now.

[MattGyverLee]

Reworked substantively based on your review. Doc dropped from 939 → 253 lines. Trying to map your inline comments to what changed:

Comment	Status
L82, L136, L338, L340, L374, L391, L421, L474, L507, L514, L539, L836, L879, L922	Fixed — most was content that duplicated windows.md or belonged in #16043
L236 (engine.groupproj discussion "unnecessary")	Fixed — deleted the section
L248 (SETX consistency)	Fixed
L545 (unnecessary CAUTION about --builder-dep-parent)	Fixed — removed
L621 (enabling→launching)	Fixed
L630 (canonical build order redundant with prompts)	Fixed — dropped the numbered list
L771 (debugging accuracy + scope)	Fixed — trimmed, dropped incorrect PDB claim, acknowledged non-CE scope
L76, L162, L854, L108, L292, .gitignore L198	See inline replies

Also one thing I got wrong in my own doc and only caught spot-checking after your review: I'd claimed all 17 Library Search Path entries mirror DELPHIINCLUDES, but that variable only defines 6 of them. The other 11 (JCL/JVCL/jedi) are IDE-only. Corrected.

Flagging for scope — this PR also drops inst\insthelper\insthelper.dproj from engine.groupproj. That path has been stale since d06f4b13d0 moved the file out of inst/. Bundled it here because it's the sort of thing CE contributors trip on via IDE Build All, but it's a drive-by. Happy to split into its own chore commit if you'd rather review it separately.

[mcdurdin]

This PR currently includes all the commits from #16043. Can you rebase onto master so we can look at just the changes for this PR?

[mcdurdin]

This is looking a lot cleaner, thank you. Suggestions in-line.

One significant change request -- and please test with a clean Delphi install before pushing, because I won't be able to do that here:

Rather than having the include paths script written in the documentation, where it will inevitably go stale, can you make any necessary updates to devtools/DevIncludePaths.pas and devtools/DevInstallPackages.pas to support Delphi 10.4/11/12 and use that to write include paths and install packages (with devtools -ai and devtools -ip, because that keeps the registration of paths and packages associated with the modules, e.g.

keyman/common/windows/delphi/ext/cef4delphi/build.sh

Lines 21 to 25 in 47e2133

	function do_build() {
	"$DEVTOOLS" -ai "$THIS_SCRIPT_PATH/source"
	delphi_msbuild packages/CEF4Delphi.dproj "//p:Platform=Win32"
	"$DEVTOOLS" -ip "$COMMON_OUTLIB/CEF4Delphi.bpl"
	}

[MattGyverLee]

Ok, @mcdurdin I'm done with these. I did a fresh build of all of the Delphi-related items. This allowed me to confirm the docs and that some of the JCL and JVCL changes are needed to build Tike.exe (at least on Windows). I also opened a couple of follow-up issues. I'm only set up to test Windows, Delphi12 AND CE together on this machine, so the 2 branch separation is a bit artificial. I can't test or speak for Delphi 12 Pro on any platform or Delphi 12 CE on Linux.

[mcdurdin]

I am out of time to continue reviewing this. I will return to it in August after I return from leave.

[MattGyverLee]

@mcdurdin , these are as clean as I can get them.

The first round of figuring this out was a brute-force session where I kept trying to build and passed Claude the error messages that surfaced, like the JCL errors and a half-dozen missing .res files. After the first JC errors were solved, I asked Claude to solve similar errors in one go. The results were not hallucinations, but some went too far (blocking some paths 100% instead of conditionally) and some defensively hit similar JC code you don't touch. The result was getting everything compiling in an hour while I was working on something else, but I wasn't trying to follow what changes were made.

I think I originally submitted the first "monoPR" as a draft and honestly didn't expect these branches to get prioritized or merged into main any time soon, I just figured it would be documentation for future reference of someone doing this "right" if they got stuck.

I can manually test most of 16043 alone via the IDE or the full build if I got a 12 trial. I can't get 10 to verify things aren't broken by my changes in 10/10CE, but I assume you and the CI have done that. You asked me to rebase both off of main, I haven't done that yet because with only 12CE, I can't test 16044 without 16043 underneath. Let me know if you still want that once this stops churning.

Once 16043 lands, I can rebase this off of main if that's still your preference. This is the clean compare (MattGyverLee/keyman@chore/windows/delphi-11-12-compat...feat/windows/delphi-ce-workflow).

Now the JC fixes are reduced to the absolute minimum for me to progress (verified by upstream changes). Removing either breaks my JC builds since I don't have installed binaries (I assume you must).

I did the full clear and rebuild during the last refactor. What I haven't tested is to uninstall Delphi and Keyman Developer and Keyman and trying to get both keyman apps running from a build without installing any precompiled binaries. I heard you say that it's possible, but haven't looked into what that entails.

I've kept the .groupproj fix on this branch because the regular build doesn't touch them, but my new CE process is MUCH faster with them.