Add comprehensive Android accessibility support following RetroArch's proven
architecture to enable screen reader functionality for visually impaired players.
The implementation uses Android's native accessibility framework (announceForAccessibility)
to integrate with TalkBack and other screen readers, providing the same accessibility
features available on Windows (Tolk) and Linux (speech-dispatcher).
## Functionality Status
✅ **Working:**
- Compiles successfully for all Android architectures
- TTS starts correctly after game data loading
- TTS interrupts speech when needed (using appropriate `force=true`)
- TTS uses Android's default text-to-speech engine
⚠️ **Known Issues (mod-related, not TTS):**
- Stats screen navigation: pressing C or equivalent button focuses on "Strength"
and prevents navigation through other attributes
- Tracking keys provide incorrect/imprecise directions, preventing proper navigation
to desired destinations
- Mod sounds (doors, chests) not playing - likely due to incorrect file paths or
Android-specific issues
See #12 for details on known issues.
## Usage Instructions
⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after
loading the game data (.mpq). The accessibility mod takes control of TTS through
Android's native API.
## Key Technical Changes
• Added JNI bridge between C++ game code and Java accessibility APIs
• Implemented thread-safe JNIEnv caching with pthread thread-local storage
• Created native methods for screen reader detection and text-to-speech
• Fixed initialization order: nativeInitAccessibility() must be called after
super.onCreate() to ensure SDL loads the native library first
• Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration
## Technical Details
- Java Layer (DevilutionXSDLActivity.java):
• isScreenReaderEnabled(): Checks TalkBack and touch exploration state
• accessibilitySpeak(): Uses announceForAccessibility() API
- JNI Bridge (Source/platform/android/android.cpp):
• GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment
• SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++
• IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code
- Platform Integration (Source/utils/screen_reader.cpp):
• Routes accessibility calls to platform-specific implementations
• Maintains consistent API across Windows, Linux, and Android
## Architecture Notes
- Follows RetroArch's accessibility implementation pattern
- Uses global JNI references for thread-safe Activity access
- Caches method IDs to avoid repeated JNI lookups
- Handles thread attachment/detachment automatically
- No external dependencies required (Android SDK only)
## Documentation
Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md
Fixes startup crash caused by calling native methods before library loading.
Verified on physical device: app launches successfully, accessibility features functional.
Closes#12
```
Add comprehensive Android accessibility support following RetroArch's proven
architecture to enable screen reader functionality for visually impaired players.
The implementation uses Android's native accessibility framework (announceForAccessibility)
to integrate with TalkBack and other screen readers, providing the same accessibility
features available on Windows (Tolk) and Linux (speech-dispatcher).
## Functionality Status
✅ **Working:**
- Compiles successfully for all Android architectures
- TTS starts correctly after game data loading
- TTS interrupts speech when needed (using appropriate `force=true`)
- TTS uses Android's default text-to-speech engine
⚠️ **Known Issues (mod-related, not TTS):**
- Stats screen navigation: pressing C or equivalent button focuses on "Strength"
and prevents navigation through other attributes
- Tracking keys provide incorrect/imprecise directions, preventing proper navigation
to desired destinations
- Mod sounds (doors, chests) not playing - likely due to incorrect file paths or
Android-specific issues
See #12 for details on known issues.
## Usage Instructions
⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after
loading the game data (.mpq). The accessibility mod takes control of TTS through
Android's native API.
## Key Technical Changes
• Added JNI bridge between C++ game code and Java accessibility APIs
• Implemented thread-safe JNIEnv caching with pthread thread-local storage
• Created native methods for screen reader detection and text-to-speech
• Fixed initialization order: nativeInitAccessibility() must be called after
super.onCreate() to ensure SDL loads the native library first
• Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration
## Technical Details
- Java Layer (DevilutionXSDLActivity.java):
• isScreenReaderEnabled(): Checks TalkBack and touch exploration state
• accessibilitySpeak(): Uses announceForAccessibility() API
- JNI Bridge (Source/platform/android/android.cpp):
• GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment
• SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++
• IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code
- Platform Integration (Source/utils/screen_reader.cpp):
• Routes accessibility calls to platform-specific implementations
• Maintains consistent API across Windows, Linux, and Android
## Architecture Notes
- Follows RetroArch's accessibility implementation pattern
- Uses global JNI references for thread-safe Activity access
- Caches method IDs to avoid repeated JNI lookups
- Handles thread attachment/detachment automatically
- No external dependencies required (Android SDK only)
## Documentation
Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md
Fixes startup crash caused by calling native methods before library loading.
Verified on physical device: app launches successfully, accessibility features functional.
Related files:
• CMake/Dependencies.cmake: Skip external dependencies for Android
• Source/CMakeLists.txt: Add Android-specific screen reader configuration
• android-project/app/build.gradle: Enable SCREEN_READER_INTEGRATION flag
Add WM_DIABTWARPUP to trigger message checks so the tracker's
entrances/exits and stairs categories detect town warps from
Catacombs L1, Caves L1, Hell L1, Hive, and Crypt back to town.
Previously only regular stairs up (WM_DIABPREVLVL) were detected.
TriggerLabelForSpeech already handles WM_DIABTWARPUP and returns
"Warp up", so labeling works automatically.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove the legacy town_npc_nav tracker system which was superseded by
the unified tracker. The old tracker was still firing on PgUp/PgDown
due to hardcoded fallback logic in PressKey().
Changes:
- Remove PgUp/PgDown fallbacks in PressKey() (keep store/chat handling)
- Remove 5 keymapper registrations (ListTownNpcs, PreviousTownNpc,
NextTownNpc, SpeakSelectedTownNpc, GoToSelectedTownNpc)
- Remove UpdateAutoWalkTownNpc() and ResetAutoWalkTownNpc() calls
- Remove migration code references in options.cpp
- Delete town_npc_nav.cpp and town_npc_nav.hpp
- Update CMakeLists.txt
Town NPC tracking is now handled by the unified tracker's NPCs category.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Upstream added new tracker categories and speech fixes inline in diablo.cpp.
Our branch has already extracted that code into separate accessibility modules,
so all 6 conflict regions are resolved by keeping our slimmed-down version.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace 7 individual navigation/tracker key registrations with 3 unified
tracker keys: PageUp (previous), PageDown (next), Home (navigate/walk).
Unbind legacy town NPC nav keys (now superseded by tracker Npcs category).
Update health key description to mention Shift+mana.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Hold Shift when pressing the health percentage key to hear mana instead.
Extract ComputePercentage helper to deduplicate the HP/mana calculation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Port upstream tracker overhaul into the modular tracker module. Adds
Npcs, Players, DungeonEntrances, Stairs, QuestLocations, and Portals
categories with collector functions, navigation, and auto-walk support.
Replaces CycleTrackerTargetKeyPressed with TrackerPageUp/PageDown/Home
key handlers that support Ctrl for category cycling and Shift for
auto-walk. Includes review fixes: nullopt guard in auto-walk, removal
of default switch clauses in favor of app_fatal, internal linkage for
non-public functions, multi-tile object approach, and distance-sorted
town dungeon entrances.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move TriggerLabelForSpeech, TownPortalLabelForSpeech, and
CollectTownDungeonTriggerIndices out of the anonymous namespace and
declare them in navigation_speech.hpp so the tracker module can reuse
them for its new DungeonEntrances and Portals categories.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When MyPlayer is null, the town NPC navigation functions now provide
spoken feedback instead of returning silently, matching the pattern used
in the tracker auto-walk code. RefreshTownNpcOrder also resets
SelectedTownNpc to avoid leaving stale state.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Guard against MyPlayer being nullptr in UpdateAutoWalkTownNpc() to
match the pattern used in UpdateAutoWalkTracker(). Without this,
MyPlayer could be dereferenced if CanPlayerTakeAction() changes its
null-handling behavior in the future.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a LogWarn call when none of the expected audio file paths for the
low HP warning sound can be loaded, making it easier to diagnose missing
or misplaced sound assets.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Guard against MyPlayer being nullptr in IsPlayerDead(),
CycleSpellHotkeys(), RefreshTownNpcOrder(), SpeakSelectedTownNpc(), and
ListTownNpcsKeyPressed() to prevent crashes during early init or after
disconnect. Also reset AutoWalkTrackerTargetId when MyPlayer is null in
the auto-walk tracker to prevent the walk loop from retrying endlessly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The variable is only used within tracker.cpp, so remove the unnecessary
extern declaration from the header and place it in the anonymous
namespace alongside the other file-local tracker state.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix copy-paste comment errors in SpellBookKeyPressed() that referred to
"inventory" instead of "spellbook", and QuestLogKeyPressed() that said
"character quest log" instead of "quest log". Correct a misleading
FindPath comment in town NPC auto-walk, and update a stale SourceX path
reference in diablo.cpp.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove ~4,042 lines of extracted function bodies and forward
declarations from diablo.cpp. Add includes for the 6 new module
headers. Update CancelAutoWalkInternal() to delegate to
ResetAutoWalkTracker() and ResetAutoWalkTownNpc().
Remove unused includes from diablo.cpp that were only needed by the
extracted code (<queue>, format_int.hpp, tile_properties.hpp).
Add the 6 new .cpp files to Source/CMakeLists.txt so they are
compiled into libdevilutionx.
Also includes post-review cleanups:
- Remove unused automap.h include from accessibility_keys.cpp
- Remove unused walk_path_speech.hpp include from diablo.cpp
- Replace .size() == 0 with .empty() in accessibility_keys.cpp
- Simplify CanAutomapBeToggledOff to direct return expression
- Remove unreachable dead code and dead FindPreferredExitTriggerIndex
function from navigation_speech.cpp
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move ~4,100 lines of accessibility code from diablo.cpp into 6 new
modules, reducing diablo.cpp from ~7,700 to ~3,660 lines.
New modules:
- controls/accessibility_keys: UI key handlers and guard functions
(IsPlayerDead, IsGameRunning, CanPlayerTakeAction, etc.)
- utils/walk_path_speech: BFS pathfinding, PosOk variants, walk
direction helpers, and path-to-speech formatting
- utils/accessibility_announcements: periodic announcements for low
HP warning sound, durability, boss health, monsters, doors
- controls/town_npc_nav: town NPC selection cycling and auto-walk
- utils/navigation_speech: exit/stairs/portal/unexplored speech and
keyboard walk key handlers
- controls/tracker: target category cycling, candidate finding,
pathfinding navigation, and auto-walk tracker
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove slot tracking when entering/exiting the belt. Now consistently
returns to row 4 column 1 when navigating up from the belt.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Always enter belt at slot 1 when navigating down from inventory
- Return to exact inventory position when navigating up from belt
- Keep left/right navigation within belt area only
- Track last inventory slot before entering belt with SlotBeforeBelt
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix inconsistent keyboard navigation when moving through multi-tile items
in the inventory. Previously, navigating into a large item (e.g., a 3x2 axe)
and then back out would exit from a different row than you entered from.
Changes:
- Add entry point tracking to remember which row/column you entered an item from
- Use tracked entry points when exiting items to maintain navigation consistency
- Add row and column position to accessibility speech (e.g., "Row 1, Column 3: Short Sword")
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use object->position instead of the archway tile position when reporting
door locations in FindFirstClosedDoorOnWalkPath and
FindFirstTrackerPathBlock. Archway tiles (negative dObject refs) resolve
to the parent door object, but the returned position was the archway
tile rather than the door itself, causing the player to stop one tile
short.
Also: move DOOR_CLOSED/DOOR_OPEN/DOOR_BLOCKED enum from objects.cpp to
objects.h so diablo.cpp can use the canonical constants directly instead
of maintaining duplicate constexpr values. Add static_assert for
MAXOBJECTS fitting in int8_t, defensive assertions in AddDoor(), and
clarify comments around door state checks.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CancelAutoWalk() now sends a walk-to-current-position command to
immediately clear the walk path buffer, instead of only preventing new
segments. Move the M key cancellation check above the CanPlayerTakeAction
guard so it works mid-walk. Add CancelAutoWalk() calls to secondary
action (D), spell action (W), and arrow-key walk so all player inputs
properly cancel an in-progress auto-walk.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix OBJ_SIGNCHEST validation mismatch (Chests case now uses
ValidateAutoWalkObjectTarget matching IsTrackedChestObject)
- Add spoken feedback for all silent early returns (bounds checks,
MyPlayer nullptr)
- M key now toggles: press again to cancel in-progress auto-walk
- A/Shift+A cancels auto-walk silently before attacking
- Auto-walk cancels when in-game menu opens
- Extract IsTrackedMonster() predicate (replaces 3 inline checks)
- Remove default: from Monsters switch cases to enable -Wswitch
exhaustiveness checking
- Add defensive re-validation in ResolveObjectTrackerTarget
- Add documentation comments
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Allow the player to automatically walk toward the currently selected
tracker target by pressing M. Supports all tracker categories: items,
chests, doors, shrines, objects, breakables, and monsters. The walk
stops when the target is within interaction range or becomes invalid.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These files had mixed CRLF/LF line endings. Normalize to CRLF to
match the .editorconfig convention for C++ source files.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
mojsior
Juan Mathews
mojsior
hidwood
ChipsAhoiMcCoy