Recent

Author Topic: Blind user and cocoa  (Read 10985 times)

rlumley

  • New member
  • *
  • Posts: 5
Blind user and cocoa
« on: November 07, 2011, 04:50:50 pm »
hi all, i'm a blind computer user on the mac.  I use voice over for my screen reader which is built-in to all macs by pressing command-f5 to activate it and stop it.  When i use lazarus with the screen reader there are many controls that say  unknown and do not function.  I believe this is because lazarus is written in carbon as all cocoa controls are accessible out of the box.  I tried recompiling lazarus with the cocoa option in the lazarus build config screen and it restarted and then crashed.  Any ideas would be greatly appreciated.

felipemdc

  • Administrator
  • Hero Member
  • *
  • Posts: 3541
Re: Blind user and cocoa
« Reply #1 on: November 07, 2011, 05:52:30 pm »
Which controls exactly work and which ones don't work?

Could you create a simple application with some basic controls like TButton, TEdit, TComboBox. Does it work with your screen reader using LCL-Carbon? And what about when building it for LCL-Cocoa?

About the IDE in Cocoa, LCL-Cocoa is not ready yet to run it.

rlumley

  • New member
  • *
  • Posts: 5
Re: Blind user and cocoa
« Reply #2 on: November 07, 2011, 06:14:31 pm »
Hi Filepe, thanks for the quick reply, its very hard for me to tell which controls aren't working as they all say unknown, all of the menus work great, some of the controls work, the most important one that does not work is the source code editor. its say unknown when i tab to it, and if i have voice over interact with it, its does nothing. some of the controls that say unknown do work once you interact with them, an example of this is the form that comes up when you start the die that lets you set the info for lazrus, compiler, and source3 code, all of these items have two unknown fields, but when i interact with the first, i do see a combo box, browse button, and an unknown item that if i interact with it is a text field that does read telling me if the item is setup correctly.  the other unknown item is a button that says start die, and it does work.  another weird problem is once i click on the item using the voice over hot key to click nothing happens until i touch a keyboard key, then it works, for example i click the start die and it sits there like i have not pressed the button, but if i press ctrl-key then  the press takes and the die starts.  i'd be willing to pay $100.00 to get these things fixed.

felipemdc

  • Administrator
  • Hero Member
  • *
  • Posts: 3541
Re: Blind user and cocoa
« Reply #3 on: November 07, 2011, 07:51:53 pm »
So it seams that the problem does not have much to do with carbon vs cocoa. The source editor is a custom control, so automatic accessibility would not work with it, we would probably need to start an Accessibility API in Lazarus which can be used for custom controls.

You mentioned many times the word "die", for example "and the die starts" what does that mean?

How can you use such text editors like the source editor with accessibility software? I really wonder how that could work. A detailed description would be interresting, mentioning what is done in each step, how do you know where you are when you move the caret, etc.

About the $100, well, I would say that the work to fix everything would be quite huge, even for only adding accessibility to the source editor it would already be a challange, so I would recommend to scale down the requisites, define them well, and increase the amount. But you can always add it to the wiki in the section for bounties, maybe someone will appear.

rlumley

  • New member
  • *
  • Posts: 5
Re: Blind user and cocoa
« Reply #4 on: November 07, 2011, 09:31:31 pm »

Hi Filepe, sorry die should have been IDE :) as for carbon versus cocoa, they say cocoa is accessible out of the box, but here is a doc on how to make carbon and custom controls accessible, http://www.filibeto.org/unix/macos/lib/dev/documentation/Carbon/Conceptual/MakingAppsAccessible/accessibility.pdf.  As for how i use the editor well, the speech reads me the entire line when i up or down arrow, and letter by letter when i left or right arrow, there are othe ways to move by word, sentence, paragraph and so on.  A blind program does need a good memory as he can't see whats on the screen at all times.  IDE's make this a bit simpler(if i could find an accessible one :)) by having hot keys to pull up lists of classes, functions, variables and the like in popup menus.  this make it much easier to navigate, i used to put comments like //myfunctionname before a function and then use search and replace to jump around.  Also in the olden days delphi for window had a pretty cool feature which was a hot key to toggle the form from graphics to text and back again, when in text mode it look like whats in your .lfm files no, and i could change object name, x, y, height, width and the like just by editing the text.  I think i can do this in lazarus, its just more tedious, i have to open lazarus, create a project, use the voice over hot key to chow open windows, and choose the form i want to modify, then i use the lazarus view menu and show components, not the pallet, just components, then i use the table view which lists components to add one to the form by pressing enter on the control. then i save the project, exit the IDE with command-q and open the form1.lfm or what ever its called in my smultron editor and modify and save it back. then go back in lazarus and build the project.

hope this helps explain things some what.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5795
    • wiki
Re: Blind user and cocoa
« Reply #5 on: November 07, 2011, 11:13:33 pm »
The problem that make sit such a huge task is the internal structure in Lazarus.

SynEdit (the Source-editor) does not talk directly to Carbon, Gtk, Qt, win, ...

- SynEdit only calls the LCL
- The LCL calls the widget-set (referring to the Lazarus widget-set code)
- There is one widget-set implementation for each framework (Carbon, Gtk, Qt, win, ...). They all have the same interface so the LCL does not need to make any difference, depending to which one it is talking.
- The widget-set makes the call to the OS/System/Framework (Carbon, Gtk, Qt, win, ...)

So to implement something like this, it must be decided how the LCL should abstract this.
Even if currently only carbon or cocoa was implemented, the LCL interface would need to be done, knowing how other accessibility frameworks do work. So they can later be added for win/QT/Gtk2

Once the LCL code exists, and an implementation for at least one widgetset, then only can the work on components like SynEdit start.

mas steindorff

  • Sr. Member
  • ****
  • Posts: 428
Re: Blind user and cocoa
« Reply #6 on: November 08, 2011, 02:56:28 am »
If I understand the issue, the popup text that describes each component on a form does not work with the text to speech software.

just brain storming here..  maybe a "form" interpreter is all that is needed.  I see a form can already be saved in XML or lfm format.  A separate editor could be used to simplify one of these formats in a way that only the fields rlumley wish to modify are presented. ie removing or simplifiing the XML tags.  The edited file could then be converted back to a visual form if need.
I can already use another text editor at the same time as long as long as I save the file before I complile with Lazarus.

thoughts? what do the FPC people use for their projects?
windows 7/10 - laz 2.0 / 1.2.6 general releases

rlumley

  • New member
  • *
  • Posts: 5
Re: Blind user and cocoa
« Reply #7 on: November 08, 2011, 07:20:57 am »
Well its a bit more than the help tag or tool tip which ever you call it :) while help tags can be very helpful to get real access it takes some more work.  As for what Filepe said he is right, except for one thing, if the lcl where fixed correctly like coco is now, there would be no need to do anything to the IDE unless it used non standard or custom controls not part of the lcl.  Filepe, how hard would it be to make simedit self coiceing just a little bit like so.. if uparro then say line; if down arrow then saline; if left arrow say character; if right arrow say character;    i'm looking at the lazarus source and may try this as a stop gap.

felipemdc

  • Administrator
  • Hero Member
  • *
  • Posts: 3541
Re: Blind user and cocoa
« Reply #8 on: November 08, 2011, 08:03:25 am »
As for what Filepe said he is right, except for one thing, if the lcl where fixed correctly like coco is now, there would be no need to do anything to the IDE unless it used non standard or custom controls not part of the lcl.

Well, the IDE uses one major custom control: SynEdit, which is the source editor. I assume that the source editor is the most important one for you, correct?

Quote
  Filepe, how hard would it be to make simedit self coiceing just a little bit like so.. if uparro then say line; if down arrow then saline; if left arrow say character; if right arrow say character;    i'm looking at the lazarus source and may try this as a stop gap.

Please read the answer from Martin, he described the process. But I'll repeat it here in the sequence that it should be made:

1> Design a new Accessibility API for the LCL and write it. The design must be very good in order to be implementable in all operating systems. Usually the best is just copying the design from Qt, because they have good ideas.
2> Write new WS functions to expose this API to the widgetsets, probably in TWSWinControl
3> Implement this new WS functions in LCL-Carbon
4> Modify SynEdit to use this accessibility API

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5795
    • wiki
Re: Blind user and cocoa
« Reply #9 on: November 08, 2011, 12:16:50 pm »
As my previous answer. The main work is outside of SynEdit. Once that is done, it will be easy to implement something for SynEdit.

However, if you whish to modify your own copy of SynEdit, to directly call carbon API (so you can at least start using it / such changes will never be part of the official release) then look for something like "ecLeft" / "ecRight" ("ecWordLeft")

"ec" stands for Editor Command (a list of them is in components\synedit\syneditkeycmds.pp)

each key-press is mapped into an ecXXX, and there is a big "case" statement. (line 6200 in synedit.pp "case Command of")

so when handling ecLeft (includes handling "ecSelLeft" Select Left), then you can just get the current line ("LineText") and access the char at "FCaret.LineBytePos"

The char behind the caret:
Code: [Select]
  if FCaret.LineBytePos <= length(LineText) then
    LineText[FCaret.LineBytePos]

Note that stepping on to the left/right, might keep the caret on the same char. A tab can span multiply positions, and the caret can be in the middle of a tab

Look at line 5032 "TCustomSynEdit.SetWordBlock" to find out how to get the current word.

A word can of course be partly outside the visible screen. SynEdit does not wrap long lines.

rlumley

  • New member
  • *
  • Posts: 5
Re: Blind user and cocoa
« Reply #10 on: November 08, 2011, 03:51:28 pm »
Martin, Filepe,  Thanks very much for taking the time to assist me with this information.  That last email was very useful Martin.  I'll try and make some mods of my own for now, then once its working so i can work :) i'll look into how to do this the correct way.  thanks very much.  how much do you think a bounty for drawing up an api for me to follow would be worth?  I'm a coded not a designer per say as usually designing works best with flow charts and such.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5795
    • wiki
Re: Blind user and cocoa
« Reply #11 on: November 08, 2011, 04:19:11 pm »
I can't advice you on the bounty. And it really depends on who will want to claim it.

Bounties often are less (far less) than the price of paying someone to develop it. But then often they are just a motivation for people who may have some other interest in doing this.

Also if you put up a bounty, you may find others that add to the bounty (the question of screen readers was discussed before).

The importance is defining what needs to be done. And also find out (or make that part of the work) what is required, so it will be accepted into the LCL.

IMHO at first you (or the taker of the bounty) need to find the different API that will be used.
You already linked the carbon api. There will be an API for windows somewhere, probably GTK and QT have one too (and cocoa).
Even if they will not be implemented, it will be good to know what data they expect, what events they may trigger, what callbacks, etc....

Then one need to divide into mandatory and optional functionality.

The LCL must provide the minimum functionality to satisfy the superset of all mandatory API functions.
The LCL functions may not be a 1 to 1 mapping. They can have any form that can later be mapped in the widegetset.

I haven't looked at any actual API, so I can't help with what it would actually look like.

---------------
About SynEdit. I will be happy to assist with any question