Text of Sikuli: Using Screenshots for GUI Automation and Testing
1. SIKULI USING SCREENSHOTS FOR GUI AUTOMATION AND TESTING
Tsung-Hsiang (Sean) Chang, Rob Miller MIT CSAIL Collaborator: Tom
Yeh 1
2. How to tell computers what to do? 2
3. case $rtval in 1) rm -f /tmp/dirip.$$ ; return ;; 255) rm -f
/tmp/dirip.$$ ; return ;; esac mfile=`cat /tmp/dirip.$$` if [ -z
$mfile ] then mfile=`pwd`/* SHELL SCRIPT else grep "*"
/tmp/dirip.$$ if [ $? -eq 1 ] then ! mfile=$mfile/* fi fi for i in
$mfile do if [ -f $i ] then ! echo "$i Delete?" >>
/tmp/finallist.$$ fi! done 3
4. 4
5. GUI AUTOMATION 5
6. 2 3 1 6
7. 7
8. click click 7
9. SIKULI SCRIPT 8
10. SIKULI TEST 9
11. OUTLINE Problem - How to refer to a GUI widget? Sikuli
Script - examples and demos Sikuli Test - demo Test By
Demonstration Support of testing practices Conclusions 10
12. HOW TO REFER TO A GUI WIDGET? 11
13. HOW TO REFER TO A GUI WIDGET? Abbot by pre-programmed
references 11
14. HOW TO REFER TO A GUI WIDGET? Abbot by pre-programmed
references by absolute locations 11
15. Use Screenshots! 12
16. TEMPLATE MATCHING click( ) 13
17. TEMPLATE MATCHING click( ) 13
18. EXAMPLE 1: EMPTY TRASH CAN 2 3 1 14
19. EX2: FACEBOOK STALKER 15
20. EX2: FACEBOOK STALKER 15
21. 16
22. RESTRICT SEARCH REGION 17
23. EX3: MUTE SPEAKERS 18
24. EX3: MUTE SPEAKERS 18
25. SPATIAL OPERATORS 19
26. SPATIAL OPERATORS 19
27. SPATIAL OPERATORS (2) 20
28. EX4: SET IP ADDRESS 21
29. VISUAL-EVENT-DRIVEN PROGRAMMING 22
30. VISUAL-EVENT-DRIVEN PROGRAMMING 22
31. SIKULI TEST 23
32. VISUAL ASSERTION Verify appearance 24
33. VISUAL ASSERTION Verify replacement 25
34. TEST BY DEMONSTRATION detect changes click 26
35. Demo 27
36. GENERATE ACTION COMMANDS mouse mouse A A b b down up down
up down up 28
37. GENERATE ACTION COMMANDS low-level events mouse mouse A A b
b down up down up down up 28
38. GENERATE ACTION COMMANDS low-level events mouse mouse A A b
b down up down up down up high-level commands click type Ab 28
39. GENERATE ACTION COMMANDS low-level events mouse mouse down
up A A b b click down up down up high-level commands click type Ab
combine with screenshots 28
42. GENERATE ASSERTIONS click 1. nd differences / manually hint
2. group together 29
43. GENERATE ASSERTIONS click 1. nd differences / manually hint
2. group together 3. combine with assertExist 29
44. GENERATE ASSERTIONS click 1. nd differences / manually hint
2. group together 3. combine with assertExist 29
45. jUNIT INTEGRATION 30
46. TEST-DRIVEN DEVELOPMENT 31
47. READABILITY OF CODE 32
48. PLATFORM INDEPENDENCE Works on ANY GUI that can be
displayed on Windows/Linux/Mac Virtual machines Remote desktop
Mobile simulators: Android, iPhone Web: Flash, HTML+Javascript
33
52. DISCUSSION Sikuli programs are written against the user
interface instead of an API UI: visible, familiar, always exists
API: faster, probably more stable 36
53. FUTURE WORK fast and accurate OCR on screen more robust
against different skins 37
54. Thanks! http://sikuli.csail.mit.edu or, just google sikuli
New Sikuli 0.10 released. Follow us on twitter @sikuli Thanks to
our funding sources: NSF and Quanta Computer (as a part of TParty
project.) 38
