最近在忙碌著CTS測試，頭大啊，各種莫名的fail，timeout，今天晚上剛好有時(shí)間來寫篇文章來記錄下；
CTS(Compatibility Test Suite)；為什么要做CTS測試：
Give your users the best possible experience with the applications they run.
When a device is compatible with Android, users can choose from among many high-quality applications.
Applications that take full advantage of Android's features are likely to perform best on compatible devices.
Make it easy for developers to write top-quality applications for your device.Developers want to streamline their applications for Android,
and this is easiest for them when they are writing for a predictable platform.
Take advantage of the Android Market.
Compatible handsets can give users access to the Android Market.
一、CTS測試之前的需要的準(zhǔn)備項(xiàng)和如何進(jìn)行測試；
做CTS測試首先得做很多項(xiàng)的配置，現(xiàn)在來仔細(xì)說下CTS的配置過程：
1、首先得安裝有adb，并且配置adb的環(huán)境變量；export PATH=$PATH:/home/myuser/adb；
2、準(zhǔn)備一個(gè)cts包，可以在源碼中通過make cts編譯，也可以在android網(wǎng)站上面下載下來（http://source.android.com/compatibility/downloads.html）；
包里面有三個(gè)文件夾，其中tools下面有測試需要的jar包以及cts-tradefed腳本用于執(zhí)行cts命令；還有一個(gè)repository文件夾，里面有一些測試計(jì)劃，測試結(jié)果也會生成在里面；在repository/testcases目錄下找到CtsDelegatingAccessibilityService.apk和/CtsDeviceAdmin.apk，并且安裝；
12 | adb install android-cts/repository/testcases/CtsDelegatingAccessibilityService.apkadb install android-cts/repository/testcases/CtsDeviceAdmin.apk |
安裝以后在setting中會多出設(shè)置項(xiàng)，需要進(jìn)行設(shè)置Settings > Accessibility > Delegating AccessibilityService，Settings > Security > Device Administrators >android.deviceadmin.cts.CtsDeviceAdmin* settings；
3、在setting中需要設(shè)置多項(xiàng)設(shè)置項(xiàng)，setting-display-sleep設(shè)置30min鐘，language設(shè)置為E文，Security中Screen lock設(shè)置NONE；
4、Cts有media stress測試，你需要在這里下下載各種分辨率的mp4視頻文件，在這里下載android-cts-media-X.Y.zip，執(zhí)行里面的copy_media.sh腳本，把視頻文件拷貝到/sdcard/test目錄下,當(dāng)然你也可以手動(dòng)push；
5、插入一張有錢的SIM卡；
都準(zhǔn)備好了接下來就可以跑CTS了，首先
1 | ./android-cts/tools/cts-tradefed |
這下就進(jìn)入了CTS測試的命令行模式了，然后執(zhí)行
1 | run cts --plan CTS |
進(jìn)行全部項(xiàng)的測試，如果你想測試某一個(gè)測試plan，可以
1 | run cts --plan <plan_name> |
這個(gè)時(shí)候等待結(jié)果就行了...漫長的等待啊......
二、如何進(jìn)行單項(xiàng)測試呢？
在Linux環(huán)境下，如果你想進(jìn)行單項(xiàng)測試，比如說某個(gè)package，class，method之類的，可以這樣
1 | run cts --package packageName |
1 | run cts --class className |
1 | run cts --class className --method methodName |
,這下就可以對單個(gè)的錯(cuò)誤進(jìn)行調(diào)試運(yùn)行了；
如果我想在windows環(huán)境下進(jìn)行CTS單項(xiàng)測試呢（不知道windows下有沒有CTS整項(xiàng)測試）？你首先需要將repositorytestcases下的所有的apk之類的文件都push到data/app下面去，然后通過
1 | adb shell pm list instrumentation |
顯示所有的Runner，這里的Runner作為下面這個(gè)命令-w以后的參數(shù)，然后通過
1 | adb shell am instrument -e class 類名#方法名 -w “Runner” |
進(jìn)行單項(xiàng)的測試；
舉個(gè)例子，比如：
123 | adb shell am instrument -e class android.accessibilityservice.cts.AccessibilityTextTraversalTest#testActionNextAndPreviousAtGranularityPageOverText -w com.android.cts.accessibilityservice/android.test.InstrumentationCtsTestRunner |
悲催的result：
1 2 3 4 5 6 7 8 910111213141516171819 | android.accessibilityservice.cts.AccessibilityTextTraversalTest:Error in testActionNextAndPreviousAtGranularityPageOverText:java.util.concurrent.TimeoutException: Expected event not received within: 5000 ms. at android.accessibilityservice.cts.AccessibilityActivityTestCase$AccessibilityInteractionBridge.executeCommandAndWaitForAccessibilityEvent(AccessibilityActivityTestCase.java:542) at android.accessibilityservice.cts.AccessibilityTextTraversalTest.testActionNextAndPreviousAtGranularityPageOverText(AccessibilityTextTraversalTest.java:1219) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:190) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:175) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661)Test results for InstrumentationCtsTestRunner=.ETime: 5.796FAILURES!!!Tests run: 1, Failures: 0, Errors: 1 |
相比windows下進(jìn)行單項(xiàng)測試，Linux下還是簡單的多了。。。
對于UiAutomatorTests的測試項(xiàng)，如何進(jìn)行單測呢？因?yàn)橹拔覀円庖妼tsUiAutomatorTests.jar push到了data/app下，我們只需執(zhí)行如下命令即可測試；
1 | uiautomator runtest /data/app/CtsUiAutomatorTests.jar -c com.android.cts.uiautomatortest.CtsUiAutomatorTest |
三、 如何針對所有的failed的項(xiàng)進(jìn)行重測呢？
首先你得通過l r命令列出測試記錄的sessionId，然后進(jìn)入respository/result/×××/testResult.xml中，首先把failed總項(xiàng)數(shù)改為0，notExecuted總項(xiàng)數(shù)改為你之前failed的項(xiàng)數(shù)；
然后通過vi的全局替換命令“1,$s/old/new/g”替換所有的result=“failed”為result=“notExecuted”，
1 | run cts --continue -session “sessionID” |
接下來就可以通過上面這個(gè)命令進(jìn)行重測了；
四、除去不需要測試的測項(xiàng)
在某些時(shí)候，當(dāng)手機(jī)沒有該項(xiàng)功能時(shí)，是否我們就不需要測試該項(xiàng)呢？比如不帶telephony的平板電腦CTS就不需要去測試telephony的項(xiàng)；
如何去除看下這段代碼就知道了：
1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738 | PackageManagerService.javavoid readPermissions() { // Read permissions from .../etc/permission directory. File libraryDir = new File(Environment.getRootDirectory(), "etc/permissions"); if (!libraryDir.exists() || !libraryDir.isDirectory()) { Slog.w(TAG, "No directory " + libraryDir + ", skipping"); return; } if (!libraryDir.canRead()) { Slog.w(TAG, "Directory " + libraryDir + " cannot be read"); return; } // Iterate over the files in the directory and scan .xml files for (File f : libraryDir.listFiles()) { // We'll read platform.xml last if (f.getPath().endsWith("etc/permissions/platform.xml")) { continue; } if (!f.getPath().endsWith(".xml")) { Slog.i(TAG, "Non-xml file " + f + " in " + libraryDir + " directory, ignoring"); continue; } if (!f.canRead()) { Slog.w(TAG, "Permissions library file " + f + " cannot be read"); continue; } readPermissionsFromXml(f); } // Read permissions from .../etc/permissions/platform.xml last so it will take precedence final File permFile = new File(Environment.getRootDirectory(), "etc/permissions/platform.xml"); readPermissionsFromXml(permFile); } |
現(xiàn)在知道了去除的方法就是把/etc/permissions下除了platform.xml的其他xml中找到telephony相關(guān)的配置文件，去除就行了，當(dāng)然其對應(yīng)的編譯前文件在framework/native/data/etc下；對其進(jìn)行修改即可；
五、CTS bug的記錄；
在此記錄一下我跟了好幾天的一個(gè)CTS bug：
1、
1 | android.media.cts.MediaPlayerTest -- testGapless1 fail junit.framework.AssertionFailedError: silence detected, please increase volume and rerun test at android.media.cts.MediaPlayerTest.testGapless(MediaPlayerTest.java:397) |
出現(xiàn)此問題的原因是EffectVisualizer.cpp中的Context狀態(tài)一直是VISUALIZER_STATE_INITIALIZED無法VISUALIZER_STATE_ACTIVE，導(dǎo)致無法走進(jìn)判斷；修改system.prop下的兩個(gè)property為false即可通過測試；PS：不同的android項(xiàng)目可能在不同的prop里面，也有可能在build.prop下；另外此選項(xiàng)是針對qcomm平臺的這種解碼方式；
Tunnel.decode use for low latency playback, it use Qualcomm HW decoder. But Tunnel decoder don't support Gapless playback, so please disable it when do CTS test.
12 | tunnel.decode=falsetunnel.audiovideo.decode=false |
當(dāng)然，你也可以
1 | getprop tunnel.decode |
然后重測看一下結(jié)果；
2、
1 | android.security.cts.PackageSignatureTest -- testPackageSignatures fail junit.framework.AssertionFailedError: These packages should not be signed with a well known key: [com.qualcomm.wifip2p.service, com.android.voicedialer, com.qualcomm.timeservice, com.qualcomm.qlogcat, com.qualcomm.atfwd, com.android.launcher, com.android.defcontainer, com.android.quicksearchbox, com.android.contacts, com.android.phone, com.android.calculator2, com.android.htmlviewer, com.android.cellbroadcastreceiver, com.android.MultiplePdpTest, com.android.providers.calendar,....... |
[reason]
using android default key to sign apk；
[Solution]
sign apk using our own key
[Change File]
buildtargetproductsecurity*
3、
1 | -- testNoSetuidTcpdump fail junit.framework.AssertionFailedError: File "/system/xbin/tcpdump" is setUID at android.security.cts.BannedFilesTest.assertNotSetugid(BannedFilesTest.java:71) |
[reason]
the permission for system/xbin/tcpdump is wrong.
[Solution]
set the correct permission to 00755 for system/xbin/tcpdump
[Change File]
system/core/include/private/android_filesystem_config.h；
4、
1 | -- testStartUsingNetworkFeature_enableHipri fail junit.framework.AssertionFailedError: Couldn't connect using hipri... at android.net.cts.ConnectivityManagerTest.testStartUsingNetworkFeature_enableHipri(ConnectivityManagerTest.java:293) |
[reason]
don't set the right APN
[Solution]
set APN and in the APN type field include "hipri"
[Change File]
N/A
5、
1 | -- testActivateGoodReceiverDisplaysActivationUi fail junit.framework.AssertionFailedError at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200) |
1234 | junit.framework.AssertionFailedError at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200) at android.admin.cts.DeviceAdminActivationTest.testActivateGoodReceiverDisplaysActivationUi(DeviceAdminActivationTest.java:102) |
在網(wǎng)上找到的一個(gè)解決辦法：
Install the "CtsDeviceAdmin.apk" and
enable only the first two
options in the security-->Deviceadministrators".
It will make the Test case "testActivateGoodReceiverDisplaysActivationUi"
PASS. Even all the 18 Test cases in this Package
(android.admin) will get PASS on Android 4.3;
但是奇怪的是這個(gè)fail報(bào)錯(cuò)的位置在源碼中找不到，DeviceAdminActivationTest,java中根本沒有200行的代碼，查看4.4的cts源碼也是一樣的；不知道是不是android-cts-4.3_r2-linux_x86-arm.zip的問題。
6、
1 | -- testUsbDebugging fail junit.framework.ComparisonFailure: expected: but was: at android.os.cts.UsbDebuggingTest.testUsbDebugging(UsbDebuggingTest.java:28) |
這個(gè)是測試secure USB debug的測項(xiàng)，需要ro.adb.secure這個(gè)property；
1 | assertEquals("1", SystemProperties.get("ro.adb.secure")); |
在main.mk文件中添加
ADDITIONAL_DEFAULT_PROPERTIES += ro.adb.secure=1即可啟用；
關(guān)于這個(gè)secure USB debugging可以參考，請務(wù)必注意修改;在開啟secure USB debugging后，如果你的機(jī)器插入usb時(shí)無法彈出authorization的彈出框，你需要更新你的adb版本，adb version：Android Debug Bridge version 1.0.31；
http://android-developers.blogspot.com/2013/02/security-enhancements-in-jelly-bean.html
7、
123456789 | android.net.cts.TrafficStatsTest -- testTrafficStatsForLocalhost fail junit.framework.AssertionFailedError: uidtxb: -1 -> -1 at android.net.cts.TrafficStatsTest.testTrafficStatsForLocalhost(TrafficStatsTest.java:136) [reason] CONFIG_UID_STAT not set[Solution] set CONFIG_UID_STAT=y[Change File] kernel/arch/arm/configs/msm8960_defconfig" |
8、
12 | org.apache.harmony.luni.tests.java.net.URLConnectionTest-- test_getOutputStream fail junit.framework.AssertionFailedError: Got different responseCode expected:<405> but was:<200> at org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getOutputStream(URLConnectionTest.java:870) |
這個(gè)CTS bug是（2014.01）最近才會出現(xiàn)的，原因是www.google.com/ie的post訪問方式的HTTP返回碼問題，無需fix，google會在最新的code中刪除掉此項(xiàng)測試；
