Add comprehensive Android accessibility support following RetroArch's proven
architecture to enable screen reader functionality for visually impaired players.
The implementation uses Android's native accessibility framework (announceForAccessibility)
to integrate with TalkBack and other screen readers, providing the same accessibility
features available on Windows (Tolk) and Linux (speech-dispatcher).
## Functionality Status
✅ **Working:**
- Compiles successfully for all Android architectures
- TTS starts correctly after game data loading
- TTS interrupts speech when needed (using appropriate `force=true`)
- TTS uses Android's default text-to-speech engine
⚠️ **Known Issues (mod-related, not TTS):**
- Stats screen navigation: pressing C or equivalent button focuses on "Strength"
and prevents navigation through other attributes
- Tracking keys provide incorrect/imprecise directions, preventing proper navigation
to desired destinations
- Mod sounds (doors, chests) not playing - likely due to incorrect file paths or
Android-specific issues
See #12 for details on known issues.
## Usage Instructions
⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after
loading the game data (.mpq). The accessibility mod takes control of TTS through
Android's native API.
## Key Technical Changes
• Added JNI bridge between C++ game code and Java accessibility APIs
• Implemented thread-safe JNIEnv caching with pthread thread-local storage
• Created native methods for screen reader detection and text-to-speech
• Fixed initialization order: nativeInitAccessibility() must be called after
super.onCreate() to ensure SDL loads the native library first
• Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration
## Technical Details
- Java Layer (DevilutionXSDLActivity.java):
• isScreenReaderEnabled(): Checks TalkBack and touch exploration state
• accessibilitySpeak(): Uses announceForAccessibility() API
- JNI Bridge (Source/platform/android/android.cpp):
• GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment
• SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++
• IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code
- Platform Integration (Source/utils/screen_reader.cpp):
• Routes accessibility calls to platform-specific implementations
• Maintains consistent API across Windows, Linux, and Android
## Architecture Notes
- Follows RetroArch's accessibility implementation pattern
- Uses global JNI references for thread-safe Activity access
- Caches method IDs to avoid repeated JNI lookups
- Handles thread attachment/detachment automatically
- No external dependencies required (Android SDK only)
## Documentation
Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md
Fixes startup crash caused by calling native methods before library loading.
Verified on physical device: app launches successfully, accessibility features functional.
Closes#12
```
Add comprehensive Android accessibility support following RetroArch's proven
architecture to enable screen reader functionality for visually impaired players.
The implementation uses Android's native accessibility framework (announceForAccessibility)
to integrate with TalkBack and other screen readers, providing the same accessibility
features available on Windows (Tolk) and Linux (speech-dispatcher).
## Functionality Status
✅ **Working:**
- Compiles successfully for all Android architectures
- TTS starts correctly after game data loading
- TTS interrupts speech when needed (using appropriate `force=true`)
- TTS uses Android's default text-to-speech engine
⚠️ **Known Issues (mod-related, not TTS):**
- Stats screen navigation: pressing C or equivalent button focuses on "Strength"
and prevents navigation through other attributes
- Tracking keys provide incorrect/imprecise directions, preventing proper navigation
to desired destinations
- Mod sounds (doors, chests) not playing - likely due to incorrect file paths or
Android-specific issues
See #12 for details on known issues.
## Usage Instructions
⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after
loading the game data (.mpq). The accessibility mod takes control of TTS through
Android's native API.
## Key Technical Changes
• Added JNI bridge between C++ game code and Java accessibility APIs
• Implemented thread-safe JNIEnv caching with pthread thread-local storage
• Created native methods for screen reader detection and text-to-speech
• Fixed initialization order: nativeInitAccessibility() must be called after
super.onCreate() to ensure SDL loads the native library first
• Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration
## Technical Details
- Java Layer (DevilutionXSDLActivity.java):
• isScreenReaderEnabled(): Checks TalkBack and touch exploration state
• accessibilitySpeak(): Uses announceForAccessibility() API
- JNI Bridge (Source/platform/android/android.cpp):
• GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment
• SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++
• IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code
- Platform Integration (Source/utils/screen_reader.cpp):
• Routes accessibility calls to platform-specific implementations
• Maintains consistent API across Windows, Linux, and Android
## Architecture Notes
- Follows RetroArch's accessibility implementation pattern
- Uses global JNI references for thread-safe Activity access
- Caches method IDs to avoid repeated JNI lookups
- Handles thread attachment/detachment automatically
- No external dependencies required (Android SDK only)
## Documentation
Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md
Fixes startup crash caused by calling native methods before library loading.
Verified on physical device: app launches successfully, accessibility features functional.
Related files:
• CMake/Dependencies.cmake: Skip external dependencies for Android
• Source/CMakeLists.txt: Add Android-specific screen reader configuration
• android-project/app/build.gradle: Enable SCREEN_READER_INTEGRATION flag
This does not fully address the issue of most everything
being a giant library but it's a start.
Note that the libraries are OBJECT libraries, so they
may (and currently do) contain references to missing
symbols.
Also fixes the implementation of transitive dependency
support for OBJECT libraries.
Juan Mathews
Gleb Mazovetskiy
staphen