Reporting unit test duration over builds

We have a set of unit tests which perform standard operations in our software, spinning up the application, creating data...
What we would like to do is create a line chart to show the running time of individual tests across builds, so we can show how code changes are altering system performance.
Is this possible? This is just a quick question before we spend resources to investigate how to do it.
Thanks in advance

As part of the Test result output we are getting TRX files. TRX files are just XMLs. You can write an XSLT transformation to present the data in the format you need. A nice thing about XSLT is that it has built-in aggregation, grouping, sorting etc capabilities
Few more options in detail given below.
http://richhewlett.com/2012/05/24/exporting-an-html-report-from-your-vstfs-test-results/
http://social.technet.microsoft.com/wiki/contents/articles/2061.how-to-create-reports-for-tfs-2010-test-results.aspx
Cheers, Senthilraj

Similar Messages

[svn:osmf:] 11045: Increasing timer interval in attempt to fix unit test on the build server.

Revision: 11045
Author:   [email protected]
Date:     2009-10-21 02:32:45 -0700 (Wed, 21 Oct 2009)
Log Message:
Increasing timer interval in attempt to fix unit test on the build server.
Modified Paths:
    osmf/trunk/framework/MediaFrameworkFlexTest/org/osmf/composition/TestParallelViewableTrai t.as

In general theory, one now has the Edit button for their posts, until someone/anyone Replies to it. I've had Edit available for weeks, as opposed to the old forum's ~ 30 mins.
That, however, is in theory. I've posted, and immediately seen something that needed editing, only to find NO Replies, yet the Edit button is no longer available, only seconds later. Still, in that same thread, I'd have the Edit button from older posts, to which there had also been no Replies even after several days/weeks. Found one that had to be over a month old, and Edit was still there.
Do not know the why/how of this behavior. At first, I thought that maybe there WAS a Reply, that "ate" my Edit button, but had not Refreshed on my screen. Refresh still showed no Replies, just no Edit either. In those cases, I just Reply and mention the [Edit].
Also, it seems that the buttons get very scrambled at times, and Refresh does not always clear that up. I end up clicking where I "think" the right button should be and hope for the best. Seems that when the buttons do bunch up they can appear at random around the page, often three atop one another, and maybe one way the heck out in left-field.
While I'm on a role, it would be nice to be able to switch between Flattened and Threaded Views on the fly. Each has a use, and having to go to Options and then come back down to the thread is a very slow process. Jive is probably incapable of this, but I can dream.
Hunt

Suggestions requested for Unit Testing process and build processes.

Hi All,
We are using WebLogic WorkShop 8.1 SP2 to build our WebApp. One thing I am trying
to get together is a "Best Practises" list for aspects of WorkShop developement,
particularly Unit Testing, Continous Build methodology, source control management
etc.I have been through the "Best Practises Guide" that comes in WorkShop help,
but it doesnt address these issues.This could help us all for future projects.
1)Could anyone give pointers on how to perform Unit Testing using either JUnit/JUnitEE
in the WorkShop realm, given that Controls cannot be accessed directly from PO
Test classes.
2)For a project of size say 5 developers ,does it make sense to have a nightly
build using tools like CruiseControl?We use CVS for our source control and its
working out pretty well, but given that we currently have no Unit Tests that can
be run after the build and that can provide some reports on what broke/what didnt?
I am sure we all would appreciate any suggestions and more questions on this topic.
Thanks,
Vik.

Hi, Chris,
can you perhaps explain your solution in greater detail. I am really curious to
find a way to test controls.
"Chris Pyrke" <[email protected]> wrote:
>
I have written (well it's a bit of a dirty hack really) something that
lends itself
to the name ControlTest (it unit tests controls). Its a blend of Junit
and Cactus
with some of the source of each brutalised a bit to get things to work
(not much
- it was a couple of hours work, when I was supposed to be doing something
else).
To write a control test you code something like...
package com.liffe;
import com.liffe.controlunit.ControlTestCase;
import controls.Example;
public class TestExample extends ControlTestCase
private Example example = null;
public void setUp() {
example = (Example)getControl("example");
public void testExample() {
this.assertNotNull(example);
String result = example.getData();
assertEquals(result, "Happy as Larry");
Other tasks required to set up a test are creating a web project with
a jpf which
needs some cut and paste code (14 lines) in its begin method and a jsp
which is
also cut and paste (5 lines). (ie create a standard web project and paste
in 2
pieces of code)
In the web project you need to create a control (A) with an instance
name of controlContainer.
(if it's called something else the pasted in code will need changing
to reflect)
In this control you need to put an instance of TestContainerImpl and
any controls
that need testing.
You then need to add a method to the control (A) that looks like…
* @common:operation
public String controlTestRun(String theSuiteClassName, boolean xsl)
container.setControl("example", example);
return container.controlTestRun(theSuiteClassName, xsl);
You need to call container.setControl for each control being tested and
the object
'container' is the instance name of the TestContainerImpl that was put
in.
There are 4 jars (junit, cactus etc) that go in the library. You will
also need
the ControlUnitBase project (or maybe just it's jar).
To use you call a URL like:
http://localhost:7001/TestWeb/Controller.jpf?test=com.liffe.TestExample
TestWeb is the name I gave to my web project - this will be different
for each
test project
com.liffe.Example is the class above and will therefore be different
for each
test case.
You can also call
http://localhost:7001/TestWeb/Controller.jpf?test=com.liffe.TestExample&xsl=true
(Note the extra &xsl=true) and the browser will (if it can) render it
more prettily.
This seems to do the job quite nicely, but there are several caveats
I would hope
someone from bea would be able to address before I start using it widely.
1) To access the control you need to create it (eg as a subcontrol in
the control
(A) above.
To get it into the test case you need to pass it round as an Object (can't
return
Control from a control operation). As it's being passed around among
Java (POJO)
classes I'm assuming that control remains in the control container context
so
this is OK. It seems to work and the Object is some form of proxy as
any control
seems to be reproxied before the control is invoked from the test case.
2) If I'm testing controls called from a JPD then they either need to
be in a
control project (and my test cases called from a Web Project) which makes
for
a large increase in project numbers (we already have this and are trying
to resist
it) To avoid this - as a process project is a brain damaged web project
I simply
perform some brain surgery and augment the process project with some
standard
files found in any old web project. this means I can call the test JPF
from a
browser. This seems nasty, is there a better way?
3) I would like to be able to deliver without the test code. At the worst
the
code can be in place but must be inacessible in a production environment.
I don't
know how best to do this - any suggestions (without creating lots of
projects,
or lots of manual effort)
If anyone has read this far I would ask the question does this seem like
the kind
of thing that would be useful?
Hopefully a future version of workshop will have something to enable
unit testing
and this hacking will be unnecessary.
Could someone from BEA tell me if this is a dangerous way to do things?
Chris
"vik" <[email protected]> wrote:
Hi All,
We are using WebLogic WorkShop 8.1 SP2 to build our WebApp. One thing
I am trying
to get together is a "Best Practises" list for aspects of WorkShop developement,
particularly Unit Testing, Continous Build methodology, source control
management
etc.I have been through the "Best Practises Guide" that comes in WorkShop
help,
but it doesnt address these issues.This could help us all for future
projects.
1)Could anyone give pointers on how to perform Unit Testing using either
JUnit/JUnitEE
in the WorkShop realm, given that Controls cannot be accessed directly
from PO
Test classes.
2)For a project of size say 5 developers ,does it make sense to have
a nightly
build using tools like CruiseControl?We use CVS for our source control
and its
working out pretty well, but given that we currently have no Unit Tests
that can
be run after the build and that can provide some reports on what broke/what
didnt?
I am sure we all would appreciate any suggestions and more questions
on this topic.
Thanks,
Vik.

Unit Testing in Report Generation

I am new to SAP-BW. looking as of how can Unit Testing be done in Reports. and any other kind of testing on reports.
any kind of information on report testing, how is it usually done would be real helpful.
thanks in advance

Hi Kavi,
As far as i have gathered knowledge in Report Unit Testing.these are the important points
you have four kind of unit testing to be done on the reports.
1.Security : This will include testing the functionality of the security module with regards to validation of user profile and that access to reports is provided based on user group profile. The users can be categorized into the following types and this information can be obtained from Report Technical Design Document.
o     View a snapshot of a report generated earlier
o     Can execute report
o     Edit the report
2.User Experice Testing: basically how the report looks to user, the fonts, the titles, kind of obvios but important stuff.
3. Data Validation Testing: To see if the data we obtain in the reports is same as the data present in the Source R/3 System. This can be done by taking some raw data from source system and cross checking with the reports. This can be done manually, I dont have any idea to automate this process.I am not sure if automation can be done. This is a painful process but, very important step and crutial in unit testing.
4. Navigational Testing: This is basically testing all the drill down function, navigational and Report to Report linkage. This info we can get from the technical design document for the reporting. and corresponding Test Scripts can be prepared from it.
As you can see The template will be consisting of all the above said columns, Entry and Exit Criteria and corresponding people who are going to perform the test.(Onsite/offsite).
For each case a testscript can be prepared and can be used multiple times whenever report has to be tested.
I do have a template.BUt i need to edit it somewat before mailing.Do tell me if u need one...or is this information sufficient.
Hope it Helps
AKB

Flash Builder 4.7 Unit Tests for Mobile Devices

I am trying to setup a simple unit test using Flash Builder 4.7. I created a Test Suite and a Test Case class.
Using the menu, I select Run > Run. I see options to select which Test Case / Methods I'd like to run.
But the strange thing is, next I have to select a launch configuration. I'd assume it would just launch in some console window and give me test results. I did what I could and have it setup to run on the AIR simulator for a XOOM device. Nothing happens I just see a blank screen with a blank bar at the top.
I've also tried right clicking the project and choosing execute flexunit tests. My FlexUnit Resulst still says 0/0.
What is the launch configuration I need to use to execute the test cases and get the results?

I solved it! After looking at example code on google, I realized that the s:ViewNavigatorApplication was missing two key options:
firstView="flexunit.flexui.FlexUnitTestRunnerUIMobile"
firstViewData="{getFirstViewData()}"
Thankfully I was able to derive this. It makes 0 since to me why this wouldn't be added to the automatically generated mxml file but the functions it utalizes does. This to me seems like a defect. If you'd like people to use automatically generated files, make sure they are functional.
Thanks

Error when running unit tests

Hi,
I'm getting the error 'Marker id <num> not found' about 9 times out of 10 when I try to run unit tests in Flash Builder 4.5 Premium. This is happening on a new (2 weeks old) spark + mx project, with the unit test classes created in Flash Builder using the built-in tools. Using Run As or Debug does not seem to make a difference to whether the error occurs or not. I have to just keep retrying until it runs correctly.
I am using the AS Code Coverage plugin and the FlexUnit 4.1.
Please help as this is driving me mad!
Thanks,
Adam

Here you go. Looks like it shrunk down the libraries image though.

Xcode service - CI Bot error when performing Unit Tests

When an Xcode Service bot attempts to perform a build with 'run unit tests' enabled, the build always fails. When looking at the log file the same error repeated many times:
xcsbuildd[1966] <Warning>: [XCSBuildOperation.m:106 7368f310 +2126ms] Preflight connection to /var/XCSControl/com.apple.XCSTestUserPreflightService failed; retrying: No such file or directory
Followed by these messages a few lines later:
xcsbuildd[1966] <Error>: [XCSBuildOperation.m:102 7368f310 +2104ms] Timed out trying to connect to CG Session; giving up after 60 seconds
xcsbuildd[1966] <Error>: [XCSOperation.m:33 7368f310 +0ms] Error Domain=XCSBuildServiceDomain Code=0 "Cancelled integration because headless test environment is required but not available after 60s" UserInfo=0x7fa952571bd0 {NSLocalizedDescription=Cancelled integration because headless test environment is required but not available after 60s}
xcsbuildd[1966] <Debug>: [XCSOperation.m:28 7368f310 +0ms] Cancelling operation: XCSBuildOperation
I've hit a brick wall with this, and can't find anything on it.
Has anyone had this problem and found a fix?

Same problem here after updating the Xcode service to the latest version. Anyone found a solution ?

[svn:osmf:] 14535: Unit tests for the syndication library.

Revision: 14535
Revision: 14535
Author:   [email protected]
Date:     2010-03-02 15:42:45 -0800 (Tue, 02 Mar 2010)
Log Message:
Unit tests for the syndication library.
Added Paths:
    osmf/trunk/libs/SyndicationTest/
    osmf/trunk/libs/SyndicationTest/.actionScriptProperties
    osmf/trunk/libs/SyndicationTest/.flexProperties
    osmf/trunk/libs/SyndicationTest/.project
    osmf/trunk/libs/SyndicationTest/SyndicationTest.mxml
    osmf/trunk/libs/SyndicationTest/html-template/
    osmf/trunk/libs/SyndicationTest/html-template/AC_OETags.js
    osmf/trunk/libs/SyndicationTest/html-template/history/
    osmf/trunk/libs/SyndicationTest/html-template/history/history.css
    osmf/trunk/libs/SyndicationTest/html-template/history/history.js
    osmf/trunk/libs/SyndicationTest/html-template/history/historyFrame.html
    osmf/trunk/libs/SyndicationTest/html-template/index.template.html
    osmf/trunk/libs/SyndicationTest/html-template/playerProductInstall.swf
    osmf/trunk/libs/SyndicationTest/libs/
    osmf/trunk/libs/SyndicationTest/libs/FlexUnit.swc
    osmf/trunk/libs/SyndicationTest/libs/FlexUnitOptional.swc
    osmf/trunk/libs/SyndicationTest/org/
    osmf/trunk/libs/SyndicationTest/org/osmf/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/SyndicationTestConstants.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/SyndicationTests.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/loader/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/loader/TestFeedLoader.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/media/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/media/TestSyndicationMediaGenerator. as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/TestAtomParser.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/TestFeedParser.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/TestRSS20Parser.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/extensions/
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/extensions/TestITunesExtensi onParser.as
    osmf/trunk/libs/SyndicationTest/org/osmf/syndication/parsers/extensions/TestMediaRSSExten sionParser.as

Hi ventura8,
>> I have added a Unit Test Library (windows) project to the solution and the tests are discovered in the test explorer and the unit tests work just fine. But i can't test the dll for Windows Phone.
>> The test are discovered in test explorer but I cannot test async methods.
>> How do I run a unit test for WP for a universal portable lib
What the async methods are. Are they the class methods in the dll?
If so, the link below shows the details of creating testing apps for windows phone.
Reference:
https://msdn.microsoft.com/en-us/library/windows/apps/dn168930(v=vs.105).aspx
>> And if i try the Unit Test App (windows Phone), it does not get discovered by test explorer. I am using VS2013 U4.
I am wondering how you created you unit test project. After you create the unit test, if you build the solution, could you get the test method in test explorer window?
Best Regards,
Edward
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click HERE to participate the survey.

[svn:osmf:] 15299: Fixing build: unit tests expect the seek to complete over a timer.

Revision: 15299
Revision: 15299
Author:   [email protected]
Date:     2010-04-09 03:04:45 -0700 (Fri, 09 Apr 2010)
Log Message:
Fixing build: unit tests expect the seek to complete over a timer. Adding this behavior on signaling the seek completion event from the newly added path.
Modified Paths:
    osmf/trunk/framework/OSMF/org/osmf/net/NetStreamSeekTrait.as

In general theory, one now has the Edit button for their posts, until someone/anyone Replies to it. I've had Edit available for weeks, as opposed to the old forum's ~ 30 mins.
That, however, is in theory. I've posted, and immediately seen something that needed editing, only to find NO Replies, yet the Edit button is no longer available, only seconds later. Still, in that same thread, I'd have the Edit button from older posts, to which there had also been no Replies even after several days/weeks. Found one that had to be over a month old, and Edit was still there.
Do not know the why/how of this behavior. At first, I thought that maybe there WAS a Reply, that "ate" my Edit button, but had not Refreshed on my screen. Refresh still showed no Replies, just no Edit either. In those cases, I just Reply and mention the [Edit].
Also, it seems that the buttons get very scrambled at times, and Refresh does not always clear that up. I end up clicking where I "think" the right button should be and hope for the best. Seems that when the buttons do bunch up they can appear at random around the page, often three atop one another, and maybe one way the heck out in left-field.
While I'm on a role, it would be nice to be able to switch between Flattened and Threaded Views on the fly. Each has a use, and having to go to Options and then come back down to the thread is a very slow process. Jive is probably incapable of this, but I can dream.
Hunt

How to genearte Unit Test reports through ANT/Command line ?

Hi,
I am using sql developer unit test feature to test my database code. I am planning to execute and generate reports by running ant script.
Is it possible to get the unit test results in any format (text,XML,HTML) after running the tests.
How to integrate report generation tasks as part of Automated builds?
Is there any command line utility do through can invoke through ant task?
Thanks,
Fernando

Fernando,
I, too, am looking to run our PL/SQL unit test suites through our automated ant build scripts. Currently, I've only been able to determine that there is a "UtUtil.bat" and "UtUtil.sh" command line utility for win and linux in /sqldeveloper/sqldeveloper/bin. However, it only take three switches:
UtUtil -run ?
UtUtil -imp ?
and
UtUtil -exp ?
While this does provide some limited value to us through automating the importing of our exported test suites and then running them as part of our builds, it doesn't help in running reports on the test runs and exporting the reports to something our build processes can consume (i.e. xml).
Also, we want to be able to run our full db build on (almost) any of our development machines and we don't want to have to have a unit test repository already preconfigured on each development db. I haven't found a utility to automate the creation of the unit test repository user and the repository, and then whena all the test suite runs have finished and the reports run, delete the repository and repository user.
We have used Quest's Code Tester product in the past and it had all of these great features that I am really hoping Oracle can either implement or expose to us.
Regards and best of luck,
Mike Sanchez

TFS Build - Unit Test is showing wrong result

HI Team,
       I have a build definition(“BDPR's Playground CI Build”) which is compiling 2 solutions and running the unit tests in them. Globally I have
6 unit tests that pass, 0 that fail and 2 that are inconclusive, which gives a total of 8 unit tests. When I run this build definition, what is reported in the build summary is “6/6 test(s) passed, 0 failed, 0 inconclusive”, which is obviously wrong. It looks
like inconclusive unit tests are completely ignored by the build system. Is this a bug or am I missing some subtle option somewhere in the build definition to get this right?
       Displaying a given build summary in Visual Studio gives less information that in the web portal. The web portal reports the number of inconclusive
unit tests (but with a wrong value – see previous item) whereas Visual Studio doesn’t report the inconclusive unit tests at all. Example: for the build “BDPR's Playground CI Build_20150409.5”, web portal reports “6/6 test(s) passed, 0 failed, 0 inconclusive”
while Visual Studio reports “6 of 6 test(s) passed”! Is this a bug or is there some misconfiguration somewhere?
Regards
Hem

Hi Hem,
Thanks for your post.
What’s the version of your TFS and VS?
Yes, as far as I know the inconclusive unit test result not show in TFS build summary by default. Bu you can view the TFS build
Diagnostic log to view which inconclusive unit tests be executed in build.
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

Unit testing and system integration testing for crystal report

Hi gurus,
I am creating crystal report by oracle procedure, will you please tell me how to do unit testing and system integration testing? appreciate very much.

The focus of this forum is report design. The actual testing of reports would be subject to your own internal policies and procedures.
Jason

Flash Builder 4.6 - How can I keep unit test classes out of the finished swc?

I have a library of code I'm building and I'm working on unit testing but I have a major issue. When my finished swc compiles no matter what I do it includes the unit test classes as part of the intellisense if you load the swc via flash. The classes aren't really in the swc since if you just try and import them they'll come up undefined. They only appear to go into the intellisense for the swc. Does anyone know how can I keep this from happening in the finished source? Currently my folder setup is like this in flash builder.
src\main - source documents for the library to get compiled
src\mock - mock class area for unit testing
src\test - unit test classes
In the project Properties panel > the first tab of my Flex Library Build path I have selected only the src\main folder for the classes to inlude in the library. No other folder paths are selected.
The "Flex Library Build Path" doesn't change my results with any setting.
Thanks,

Mel Riffe,
Here's a Help topic about compiler options in Flash Builder: http://help.adobe.com/en_US/flashbuilder/using/WSe4e4b720da9dedb524b8220812e5611f28f-7fe7. html
For information on using mxmlc, the application compiler, to compile SWF files from your ActionScript and MXML source files, you can see: http://help.adobe.com/en_US/flex/using/WS2db454920e96a9e51e63e3d11c0bf69084-7fcc.html
Hope this helps,
Mallika Yelandur
Flash Builder Community Help & Learning
Adobe Systems Incorporated

Flex Builder 3 Debugger hanging unit tests

If I try to run my Unit Tests in debug mode from Flex Builder 3, FF and IE just hang until I cancel the debugger. Is this expected?

Nope. I developed the whole core in debug mode of Flex Builder 3, so I know it works there.
Mike

Unit Test in SQL Developer 2.1: Automated Builds

Hi,
I am interested to know if the new Unit Testing framework can be accessed via API so the test execution is initiated from automated build process.
Regards,
Vadim

I am having a problem with the unit testing command line.
I am attempting to run the unit testing using the command line interface.
I can connect to UNIT_TEST_REPOS schema in SQL developer.
I am successfully running units test and suites in SQL developer.
UNIT_TEST_REPOS, RCSV1 and DEVER users are granted on the UT_REPO_USER role and UNIT_TEST_REPOS and DEVER on the UT_REPO_ADMINISTRATOR.
The following commands result in an error box saying "No Repository was found on the selected connection, you need to create a repository." (The HELP button apparently does nothing. The OK button closes the box.)
C:\Program Files\sqldeveloper\sqldeveloper\bin>UtUtil -exp -test -name RCSV1_RCS_SECURITY.GET_LDAP_BASE -repo unit_test_repos -file c:\ut_xml\test.xml
Unable to open repository
C:\Program Files\sqldeveloper\sqldeveloper\bin>UtUtil -run -test -name RCSV1_RCS_SECURITY.GET_LDAP_BASE -repo unit_test_repos -db dever
Unable to open repository
C:\Program Files\sqldeveloper\sqldeveloper\bin>UtUtil -run -test -name RCSV1_RCS_SECURITY.GET_LDAP_BASE -repo dever -db dever
Unable to open repository
I would guess that I am not supplying the correct connection info.
My last comment triggered an idea. It turns out that the connection names required are those connections named in SQL developer. In my case, they are not the same as the schema names. The following command worked as advertised.
C:\Program Files\sqldeveloper\sqldeveloper\bin>UtUtil -run -test -name RCSV1_RCS_SECURITY.GET_LDAP_BASE -repo UNIT_TEST -db DeverLocal
The ANT target is
<target name="UnitTests">
<exec executable="cmd" dir="${sqldev.bin.dir}">
<arg value="/c"/>
<arg value="UtUtil -run -suite -name RCSV1 -repo UNIT_TEST -db DeverLocal"/>
</exec>
</target>
Regards,
Bill

Reporting unit test duration over builds

Similar Messages

Maybe you are looking for