mirror/devilutionX - devilutionX - xhrpb gitea

Author	SHA1	Message	Date
Juan Mathews	7ee96e45b6	feat(android): Implement native accessibility support with TalkBack integration Add comprehensive Android accessibility support following RetroArch's proven architecture to enable screen reader functionality for visually impaired players. The implementation uses Android's native accessibility framework (announceForAccessibility) to integrate with TalkBack and other screen readers, providing the same accessibility features available on Windows (Tolk) and Linux (speech-dispatcher). ## Functionality Status ✅ Working: - Compiles successfully for all Android architectures - TTS starts correctly after game data loading - TTS interrupts speech when needed (using appropriate `force=true`) - TTS uses Android's default text-to-speech engine ⚠️ Known Issues (mod-related, not TTS): - Stats screen navigation: pressing C or equivalent button focuses on "Strength" and prevents navigation through other attributes - Tracking keys provide incorrect/imprecise directions, preventing proper navigation to desired destinations - Mod sounds (doors, chests) not playing - likely due to incorrect file paths or Android-specific issues See #12 for details on known issues. ## Usage Instructions ⚠️ IMPORTANT: To play, you must suspend the screen reader (TalkBack) after loading the game data (.mpq). The accessibility mod takes control of TTS through Android's native API. ## Key Technical Changes • Added JNI bridge between C++ game code and Java accessibility APIs • Implemented thread-safe JNIEnv caching with pthread thread-local storage • Created native methods for screen reader detection and text-to-speech • Fixed initialization order: nativeInitAccessibility() must be called after super.onCreate() to ensure SDL loads the native library first • Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration ## Technical Details - Java Layer (DevilutionXSDLActivity.java): • isScreenReaderEnabled(): Checks TalkBack and touch exploration state • accessibilitySpeak(): Uses announceForAccessibility() API - JNI Bridge (Source/platform/android/android.cpp): • GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment • SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++ • IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code - Platform Integration (Source/utils/screen_reader.cpp): • Routes accessibility calls to platform-specific implementations • Maintains consistent API across Windows, Linux, and Android ## Architecture Notes - Follows RetroArch's accessibility implementation pattern - Uses global JNI references for thread-safe Activity access - Caches method IDs to avoid repeated JNI lookups - Handles thread attachment/detachment automatically - No external dependencies required (Android SDK only) ## Documentation Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md Fixes startup crash caused by calling native methods before library loading. Verified on physical device: app launches successfully, accessibility features functional. Closes #12 ```	1 month ago
Juan Mathews	fc45320e6a	feat(android): Implement native accessibility support with TalkBack integration Add comprehensive Android accessibility support following RetroArch's proven architecture to enable screen reader functionality for visually impaired players. The implementation uses Android's native accessibility framework (announceForAccessibility) to integrate with TalkBack and other screen readers, providing the same accessibility features available on Windows (Tolk) and Linux (speech-dispatcher). ## Functionality Status ✅ Working: - Compiles successfully for all Android architectures - TTS starts correctly after game data loading - TTS interrupts speech when needed (using appropriate `force=true`) - TTS uses Android's default text-to-speech engine ⚠️ Known Issues (mod-related, not TTS): - Stats screen navigation: pressing C or equivalent button focuses on "Strength" and prevents navigation through other attributes - Tracking keys provide incorrect/imprecise directions, preventing proper navigation to desired destinations - Mod sounds (doors, chests) not playing - likely due to incorrect file paths or Android-specific issues See #12 for details on known issues. ## Usage Instructions ⚠️ IMPORTANT: To play, you must suspend the screen reader (TalkBack) after loading the game data (.mpq). The accessibility mod takes control of TTS through Android's native API. ## Key Technical Changes • Added JNI bridge between C++ game code and Java accessibility APIs • Implemented thread-safe JNIEnv caching with pthread thread-local storage • Created native methods for screen reader detection and text-to-speech • Fixed initialization order: nativeInitAccessibility() must be called after super.onCreate() to ensure SDL loads the native library first • Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration ## Technical Details - Java Layer (DevilutionXSDLActivity.java): • isScreenReaderEnabled(): Checks TalkBack and touch exploration state • accessibilitySpeak(): Uses announceForAccessibility() API - JNI Bridge (Source/platform/android/android.cpp): • GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment • SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++ • IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code - Platform Integration (Source/utils/screen_reader.cpp): • Routes accessibility calls to platform-specific implementations • Maintains consistent API across Windows, Linux, and Android ## Architecture Notes - Follows RetroArch's accessibility implementation pattern - Uses global JNI references for thread-safe Activity access - Caches method IDs to avoid repeated JNI lookups - Handles thread attachment/detachment automatically - No external dependencies required (Android SDK only) ## Documentation Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md Fixes startup crash caused by calling native methods before library loading. Verified on physical device: app launches successfully, accessibility features functional. Related files: • CMake/Dependencies.cmake: Skip external dependencies for Android • Source/CMakeLists.txt: Add Android-specific screen reader configuration • android-project/app/build.gradle: Enable SCREEN_READER_INTEGRATION flag	1 month ago
mojsior	a5d43059e4	Diablo Access: accessibility improvements	2 months ago
Anders Jenbo	d725fdb4f3	Add screen reader support	3 years ago