C0 code coverage information
Generated on Wed Oct 07 08:33:59 -0700 2009 with rcov 0.8.2.1
Code reported as executed by Ruby looks like this...
and this: this line is also marked as covered.
Lines considered as run by rcov, but not reported by Ruby, look like this,
and this: these lines were inferred by rcov (using simple heuristics).
Finally, here's a line marked as not executed.
1 # Licensed to the Apache Software Foundation (ASF) under one or more
2 # contributor license agreements. See the NOTICE file distributed with this
3 # work for additional information regarding copyright ownership. The ASF
4 # licenses this file to you under the Apache License, Version 2.0 (the
5 # "License"); you may not use this file except in compliance with the License.
6 # You may obtain a copy of the License at
7 #
8 # http://www.apache.org/licenses/LICENSE-2.0
9 #
10 # Unless required by applicable law or agreed to in writing, software
11 # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
12 # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
13 # License for the specific language governing permissions and limitations under
14 # the License.
15
16
17 require 'buildr/core/project'
18 require 'buildr/core/build'
19 require 'buildr/core/compile'
20
21
22 module Buildr
23
24 # The underlying test framework used by TestTask.
25 # To add a new test framework, extend TestFramework::Base and add your framework using:
26 # Buildr::TestFramework << MyFramework
27 module TestFramework
28
29 class << self
30
31 # Returns true if the specified test framework exists.
32 def has?(name)
33 frameworks.any? { |framework| framework.to_sym == name.to_sym }
34 end
35
36 # Select a test framework by its name.
37 def select(name)
38 frameworks.detect { |framework| framework.to_sym == name.to_sym }
39 end
40
41 # Identify which test framework applies for this project.
42 def select_from(project)
43 # Look for a suitable test framework based on the compiled language,
44 # which may return multiple candidates, e.g. JUnit and TestNG for Java.
45 # Pick the one used in the parent project, if not, whichever comes first.
46 candidates = frameworks.select { |framework| framework.applies_to?(project) }
47 parent = project.parent
48 parent && candidates.detect { |framework| framework.to_sym == parent.test.framework } || candidates.first
49 end
50
51 # Adds a test framework to the list of supported frameworks.
52 #
53 # For example:
54 # Buildr::TestFramework << Buildr::JUnit
55 def add(framework)
56 @frameworks ||= []
57 @frameworks |= [framework]
58 end
59 alias :<< :add
60
61 # Returns a list of available test frameworks.
62 def frameworks
63 @frameworks ||= []
64 end
65
66 end
67
68 # Base class for all test frameworks, with common functionality. Extend and over-ride as you see fit
69 # (see JUnit as an example).
70 class Base
71
72 class << self
73
74 # The framework's identifier (e.g. :junit). Inferred from the class name.
75 def to_sym
76 @symbol ||= name.split('::').last.downcase.to_sym
77 end
78
79 # Returns true if this framework applies to the current project. For example, JUnit returns
80 # true if the tests are written in Java.
81 def applies_to?(project)
82 raise 'Not implemented'
83 end
84
85 # Returns a list of dependencies for this framework. Default is an empty list,
86 # override to add dependencies.
87 def dependencies
88 @dependencies ||= []
89 end
90
91 end
92
93 # Construct a new test framework with the specified options. Note that options may
94 # change before the framework is run.
95 def initialize(test_task, options)
96 @options = options
97 @task = test_task
98 end
99
100 # Options for this test framework.
101 attr_reader :options
102 # The test task we belong to
103 attr_reader :task
104
105 # Returns a list of dependenices for this framework. Defaults to calling the #dependencies
106 # method on the class.
107 def dependencies
108 self.class.dependencies
109 end
110
111 # TestTask calls this method to return a list of test names that can be run in this project.
112 # It then applies the include/exclude patterns to arrive at the list of tests that will be
113 # run, and call the #run method with that list.
114 #
115 # This method should return a list suitable for using with the #run method, but also suitable
116 # for the user to manage. For example, JUnit locates all the tests in the test.compile.target
117 # directory, and returns the class names, which are easier to work with than file names.
118 def tests(dependencies)
119 raise 'Not implemented'
120 end
121
122 # TestTask calls this method to run the named (and only those) tests. This method returns
123 # the list of tests that ran successfully.
124 def run(tests, dependencies)
125 raise 'Not implemented'
126 end
127
128 end
129
130 end
131
132
133 # The test task controls the entire test lifecycle.
134 #
135 # You can use the test task in three ways. You can access and configure specific test tasks,
136 # e.g. enhance the #compile task, or run code during #setup/#teardown.
137 #
138 # You can use convenient methods that handle the most common settings. For example,
139 # add dependencies using #with, or include only specific tests using #include.
140 #
141 # You can also enhance this task directly. This task will first execute the #compile task, followed
142 # by the #setup task, run the unit tests, any other enhancements, and end by executing #teardown.
143 #
144 # The test framework is determined based on the available test files, for example, if the test
145 # cases are written in Java, then JUnit is selected as the test framework. You can also select
146 # a specific test framework, for example, to use TestNG instead of JUnit:
147 # test.using :testng
148 class TestTask < Rake::Task
149
150 class << self
151
152 # Used by the local test and integration tasks to
153 # a) Find the local project(s),
154 # b) Find all its sub-projects and narrow down to those that have either unit or integration tests,
155 # c) Run all the (either unit or integration) tests, and
156 # d) Ignore failure if necessary.
157 def run_local_tests(integration) #:nodoc:
158 Project.local_projects do |project|
159 # !(foo ^ bar) tests for equality and accepts nil as false (and select is less obfuscated than reject on ^).
160 projects = ([project] + project.projects).select { |project| !(project.test.options[:integration] ^ integration) }
161 projects.each do |project|
162 info "Testing #{project.name}"
163 begin
164 project.test.invoke
165 rescue
166 raise unless Buildr.options.test == :all
167 end
168 end
169 end
170 end
171
172 # Used by the test/integration rule to only run tests that match the specified names.
173 def only_run(tests) #:nodoc:
174 tests = tests.map { |name| name =~ /\*/ ? name : "*#{name}*" }
175 # Since the tests may reside in a sub-project, we need to set the include/exclude pattern on
176 # all sub-projects, but only invoke test on the local project.
177 Project.projects.each { |project| project.test.send :only_run, tests }
178 end
179 end
180
181 # Default options already set on each test task.
182 def default_options
183 { :fail_on_failure=>true, :fork=>:once, :properties=>{}, :environment=>{} }
184 end
185
186 def initialize(*args) #:nodoc:
187 super
188 @dependencies = FileList[]
189 @include = []
190 @exclude = []
191 @forced_need = false
192 parent_task = Project.parent_task(name)
193 if parent_task.respond_to?(:options)
194 @options = OpenObject.new { |hash, key| hash[key] = parent_task.options[key].clone rescue hash[key] = parent_task.options[key] }
195 else
196 @options = OpenObject.new(default_options)
197 end
198 enhance [application.buildfile.name] do
199 run_tests if framework
200 end
201 end
202
203 # The dependencies used for running the tests. Includes the compiled files (compile.target)
204 # and their dependencies. Will also include anything you pass to #with, shared between the
205 # testing compile and run dependencies.
206 attr_reader :dependencies
207
208 # *Deprecated*: Use dependencies instead.
209 def classpath
210 Buildr.application.deprecated 'Use dependencies instead.'
211 dependencies
212 end
213
214 # *Deprecated*: Use dependencies= instead.
215 def classpath=(artifacts)
216 Buildr.application.deprecated 'Use dependencies= instead.'
217 self.dependencies = artifacts
218 end
219
220 def execute(args) #:nodoc:
221 if Buildr.options.test == false
222 info "Skipping tests for #{project.name}"
223 return
224 end
225 setup.invoke
226 begin
227 super
228 rescue RuntimeError
229 raise if options[:fail_on_failure]
230 ensure
231 teardown.invoke
232 end
233 end
234
235 # :call-seq:
236 # compile(*sources) => CompileTask
237 # compile(*sources) { |task| .. } => CompileTask
238 #
239 # The compile task is similar to the Project's compile task. However, it compiles all
240 # files found in the src/test/{source} directory into the target/test/{code} directory.
241 # This task is executed by the test task before running any tests.
242 #
243 # Once the project definition is complete, all dependencies from the regular
244 # compile task are copied over, so you only need to specify dependencies
245 # specific to your tests. You can do so by calling #with on the test task.
246 # The dependencies used here are also copied over to the junit task.
247 def compile(*sources, &block)
248 @project.task('test:compile').from(sources).enhance &block
249 end
250
251 # :call-seq:
252 # resources(*prereqs) => ResourcesTask
253 # resources(*prereqs) { |task| .. } => ResourcesTask
254 #
255 # Executes by the #compile task to copy resource files over. See Project#resources.
256 def resources(*prereqs, &block)
257 @project.task('test:resources').enhance prereqs, &block
258 end
259
260 # :call-seq:
261 # setup(*prereqs) => task
262 # setup(*prereqs) { |task| .. } => task
263 #
264 # Returns the setup task. The setup task is executed at the beginning of the test task,
265 # after compiling the test files.
266 def setup(*prereqs, &block)
267 @project.task('test:setup').enhance prereqs, &block
268 end
269
270 # :call-seq:
271 # teardown(*prereqs) => task
272 # teardown(*prereqs) { |task| .. } => task
273 #
274 # Returns the teardown task. The teardown task is executed at the end of the test task.
275 def teardown(*prereqs, &block)
276 @project.task('test:teardown').enhance prereqs, &block
277 end
278
279 # :call-seq:
280 # with(*specs) => self
281 #
282 # Specify artifacts (specs, tasks, files, etc) to include in the dependencies list
283 # when compiling and running tests.
284 def with(*artifacts)
285 @dependencies |= Buildr.artifacts(artifacts.flatten).uniq
286 compile.with artifacts
287 self
288 end
289
290 # Returns various test options.
291 attr_reader :options
292
293 # :call-seq:
294 # using(options) => self
295 #
296 # Sets various test options from a hash and returns self. For example:
297 # test.using :fork=>:each, :properties=>{ 'url'=>'http://localhost:8080' }
298 #
299 # Can also be used to select the test framework, or to run these tests as
300 # integration tests. For example:
301 # test.using :testng
302 # test.using :integration
303 #
304 # The :fail_on_failure option specifies whether the task should fail if
305 # any of the tests fail (default), or should report the failures but continue
306 # running the build (when set to false).
307 #
308 # All other options depend on the capability of the test framework. These options
309 # should be used the same way across all frameworks that support them:
310 # * :fork -- Fork once for each project (:once, default), for each test in each
311 # project (:each), or don't fork at all (false).
312 # * :properties -- Properties pass to the test, e.g. in Java as system properties.
313 # * :environment -- Environment variables. This hash is made available in the
314 # form of environment variables.
315 def using(*args)
316 args.pop.each { |key, value| options[key.to_sym] = value } if Hash === args.last
317 args.each do |name|
318 if TestFramework.has?(name)
319 self.framework = name
320 elsif name == :integration
321 options[:integration] = true
322 else
323 Buildr.application.deprecated "Please replace with using(:#{name}=>true)"
324 options[name.to_sym] = true
325 end
326 end
327 self
328 end
329
330 # :call-seq:
331 # include(*names) => self
332 #
333 # Include only the specified tests. Unless specified, the default is to include
334 # all tests identified by the test framework. This method accepts multiple arguments
335 # and returns self.
336 #
337 # Tests are specified by their full name, but you can use glob patterns to select
338 # multiple tests, for example:
339 # test.include 'com.example.FirstTest' # FirstTest only
340 # test.include 'com.example.*' # All tests under com/example
341 # test.include 'com.example.Module*' # All tests starting with Module
342 # test.include '*.{First,Second}Test' # FirstTest, SecondTest
343 def include(*names)
344 @include += names
345 self
346 end
347
348 # :call-seq:
349 # exclude(*names) => self
350 #
351 # Exclude the specified tests. This method accepts multiple arguments and returns self.
352 # See #include for the type of arguments you can use.
353 def exclude(*names)
354 @exclude += names
355 self
356 end
357
358 # *Deprecated*: Use tests instead.
359 def classes
360 Buildr.application.deprecated 'Call tests instead of classes'
361 tests
362 end
363
364 # After running the task, returns all tests selected to run, based on availability and include/exclude pattern.
365 attr_reader :tests
366 # After running the task, returns all the tests that failed, empty array if all tests passed.
367 attr_reader :failed_tests
368 # After running the task, returns all the tests that passed, empty array if no tests passed.
369 attr_reader :passed_tests
370
371 # :call-seq:
372 # framework => symbol
373 #
374 # Returns the test framework, e.g. :junit, :testng.
375 def framework
376 unless @framework
377 # Start with all frameworks that apply (e.g. JUnit and TestNG for Java),
378 # and pick the first (default) one, unless already specified in parent project.
379 candidates = TestFramework.frameworks.select { |cls| cls.applies_to?(@project) }
380 candidate = @project.parent && candidates.detect { |framework| framework.to_sym == @project.parent.test.framework } ||
381 candidates.first
382 self.framework = candidate if candidate
383 end
384 @framework && @framework.class.to_sym
385 end
386
387 # :call-seq:
388 # report_to => file
389 #
390 # Test frameworks that can produce reports, will write them to this directory.
391 #
392 # This is framework dependent, so unless you use the default test framework, call this method
393 # after setting the test framework.
394 def report_to
395 @report_to ||= file(@project.path_to(:reports, framework)=>self)
396 end
397
398 # The path to the file that stores the time stamp of the last successful test run.
399 def last_successful_run_file #:nodoc:
400 File.join(report_to.to_s, 'last_successful_run')
401 end
402
403 # The time stamp of the last successful test run. Or Rake::EARLY if no successful test run recorded.
404 def timestamp #:nodoc:
405 File.exist?(last_successful_run_file) ? File.mtime(last_successful_run_file) : Rake::EARLY
406 end
407
408 # The project this task belongs to.
409 attr_reader :project
410
411 protected
412
413 def associate_with(project)
414 @project = project
415 end
416
417 def framework=(name)
418 cls = TestFramework.select(name) or raise ArgumentError, "No #{name} test framework available. Did you install it?"
419 #cls.inherit_options.reject { |name| options.has_key?(name) }.
420 # each { |name| options[name] = @parent_task.options[name] } if @parent_task.respond_to?(:options)
421 @framework = cls.new(self, options)
422 # Test framework dependency.
423 with @framework.dependencies
424 end
425
426 # :call-seq:
427 # include?(name) => boolean
428 #
429 # Returns true if the specified test name matches the inclusion/exclusion pattern. Used to determine
430 # which tests to execute.
431 def include?(name)
432 (@include.empty? || @include.any? { |pattern| File.fnmatch(pattern, name) }) &&
433 !@exclude.any? { |pattern| File.fnmatch(pattern, name) }
434 end
435
436 # Runs the tests using the selected test framework.
437 def run_tests
438 dependencies = Buildr.artifacts(self.dependencies).map(&:to_s).uniq
439 rm_rf report_to.to_s
440 @tests = @framework.tests(dependencies).select { |test| include?(test) }.sort
441 if @tests.empty?
442 @passed_tests, @failed_tests = [], []
443 else
444 info "Running tests in #{@project.name}"
445 begin
446 # set the baseDir system property if not set
447 @framework.options[:properties] = { 'baseDir' => @project.test.compile.target.to_s }.merge(@framework.options[:properties] || {})
448 @passed_tests = @framework.run(@tests, dependencies)
449 rescue Exception=>ex
450 error "Test framework error: #{ex.message}"
451 error ex.backtrace.join("\n") if Buildr.application.options.trace
452 @passed_tests = []
453 end
454 @failed_tests = @tests - @passed_tests
455 unless @failed_tests.empty?
456 error "The following tests failed:\n#{@failed_tests.join("\n")}"
457 fail 'Tests failed!'
458 end
459 end
460 record_successful_run unless @forced_need
461 end
462
463 # Call this method when a test run is successful to record the current system time.
464 def record_successful_run #:nodoc:
465 mkdir_p report_to.to_s
466 touch last_successful_run_file
467 end
468
469 # Limit running tests to specific list.
470 def only_run(tests)
471 @include = Array(tests)
472 @exclude.clear
473 @forced_need = true
474 end
475
476 def invoke_prerequisites(args, chain) #:nodoc:
477 @prerequisites |= FileList[@dependencies.uniq]
478 super
479 end
480
481 def needed? #:nodoc:
482 latest_prerequisite = @prerequisites.map { |p| application[p, @scope] }.max { |a,b| a.timestamp<=>b.timestamp }
483 needed = (timestamp == Rake::EARLY) || latest_prerequisite.timestamp > timestamp
484 trace "Testing#{needed ? ' ' : ' not '}needed. " +
485 "Latest prerequisite change: #{latest_prerequisite.timestamp} (#{latest_prerequisite.to_s}). " +
486 "Last successful test run: #{timestamp}."
487 return needed || @forced_need || Buildr.options.test == :all
488 end
489 end
490
491
492 # The integration tests task. Buildr has one such task (see Buildr#integration) that runs
493 # all tests marked with :integration=>true, and has a setup/teardown tasks separate from
494 # the unit tests.
495 class IntegrationTestsTask < Rake::Task
496
497 def initialize(*args) #:nodoc:
498 super
499 @setup = task("#{name}:setup")
500 @teardown = task("#{name}:teardown")
501 enhance do
502 info 'Running integration tests...'
503 TestTask.run_local_tests true
504 end
505 end
506
507 def execute(args) #:nodoc:
508 setup.invoke
509 begin
510 super
511 ensure
512 teardown.invoke
513 end
514 end
515
516 # :call-seq:
517 # setup(*prereqs) => task
518 # setup(*prereqs) { |task| .. } => task
519 #
520 # Returns the setup task. The setup task is executed before running the integration tests.
521 def setup(*prereqs, &block)
522 @setup.enhance prereqs, &block
523 end
524
525 # :call-seq:
526 # teardown(*prereqs) => task
527 # teardown(*prereqs) { |task| .. } => task
528 #
529 # Returns the teardown task. The teardown task is executed after running the integration tests.
530 def teardown(*prereqs, &block)
531 @teardown.enhance prereqs, &block
532 end
533
534 end
535
536
537 # Methods added to Project to support compilation and running of tests.
538 module Test
539
540 include Extension
541
542 first_time do
543 desc 'Run all tests'
544 task('test') { TestTask.run_local_tests false }
545
546 # This rule takes a suffix and runs that tests in the current project. For example;
547 # buildr test:MyTest
548 # will run the test com.example.MyTest, if such a test exists for this project.
549 #
550 # If you want to run multiple test, separate them with a comma. You can also use glob
551 # (* and ?) patterns to match multiple tests, see the TestTask#include method.
552 rule /^test:.*$/ do |task|
553 # The map works around a JRuby bug whereby the string looks fine, but fails in fnmatch.
554 TestTask.only_run task.name.scan(/test:(.*)/)[0][0].split(',').map { |t| "#{t}" }
555 task('test').invoke
556 end
557
558 IntegrationTestsTask.define_task('integration')
559
560 # Similar to test:[pattern] but for integration tests.
561 rule /^integration:.*$/ do |task|
562 unless task.name.split(':')[1] =~ /^(setup|teardown)$/
563 # The map works around a JRuby bug whereby the string looks fine, but fails in fnmatch.
564 TestTask.only_run task.name[/integration:(.*)/, 1].split(',').map { |t| "#{t}" }
565 task('integration').invoke
566 end
567 end
568
569 end
570
571 before_define do |project|
572 # Define a recursive test task, and pass it a reference to the project so it can discover all other tasks.
573 test = TestTask.define_task('test')
574 test.send :associate_with, project
575
576 # Similar to the regular resources task but using different paths.
577 resources = ResourcesTask.define_task('test:resources')
578 resources.send :associate_with, project, :test
579 project.path_to(:source, :test, :resources).tap { |dir| resources.from dir if File.exist?(dir) }
580
581 # Similar to the regular compile task but using different paths.
582 compile = CompileTask.define_task('test:compile'=>[project.compile, resources])
583 compile.send :associate_with, project, :test
584 test.enhance [compile]
585
586 # Define these tasks once, otherwise we may get a namespace error.
587 test.setup ; test.teardown
588 end
589
590 after_define do |project|
591 test = project.test
592 # Dependency on compiled tests and resources. Dependencies added using with.
593 test.dependencies.concat [test.compile.target, test.resources.target].compact
594 # Dependency on compiled code, its dependencies and resources.
595 test.with [project.compile.target, project.resources.target].compact
596 test.with project.compile.dependencies
597 # Picking up the test frameworks adds further dependencies.
598 test.framework
599
600 project.build test unless test.options[:integration]
601
602 project.clean do
603 rm_rf test.compile.target.to_s if test.compile.target
604 rm_rf test.report_to.to_s
605 end
606 end
607
608
609 # :call-seq:
610 # test(*prereqs) => TestTask
611 # test(*prereqs) { |task| .. } => TestTask
612 #
613 # Returns the test task. The test task controls the entire test lifecycle.
614 #
615 # You can use the test task in three ways. You can access and configure specific
616 # test tasks, e.g. enhance the compile task by calling test.compile, setup for
617 # the tests by enhancing test.setup and so forth.
618 #
619 # You can use convenient methods that handle the most common settings. For example,
620 # add dependencies using test.with, or include only specific tests using test.include.
621 #
622 # You can also enhance this task directly. This method accepts a list of arguments
623 # that are used as prerequisites and an optional block that will be executed by the
624 # test task.
625 #
626 # This task compiles the project and the tests (in that order) before running any tests.
627 # It execute the setup task, runs all the tests, any enhancements, and ends with the
628 # teardown tasks.
629 def test(*prereqs, &block)
630 task('test').enhance prereqs, &block
631 end
632
633 # :call-seq:
634 # integration { |task| .... }
635 # integration => IntegrationTestTask
636 #
637 # Use this method to return the integration tests task, or enhance it with a block to execute.
638 #
639 # There is one integration tests task you can execute directly, or as a result of running the package
640 # task (or tasks that depend on it, like install and upload). It contains all the tests marked with
641 # :integration=>true, all other tests are considered unit tests and run by the test task before packaging.
642 # So essentially: build=>test=>packaging=>integration=>install/upload.
643 #
644 # You add new tests from projects that define integration tests using the regular test task,
645 # but with the following addition:
646 # test.using :integration
647 #
648 # Use this method to enhance the setup and teardown tasks that are executed before (and after) all
649 # integration tests are run, for example, to start a Web server or create a database.
650 def integration(*deps, &block)
651 Rake::Task['rake:integration'].enhance deps, &block
652 end
653
654 end
655
656
657 # :call-seq:
658 # integration { |task| .... }
659 # integration => IntegrationTestTask
660 #
661 # Use this method to return the integration tests task.
662 def integration(*deps, &block)
663 Rake::Task['rake:integration'].enhance deps, &block
664 end
665
666 class Options
667
668 # Runs tests after the build when true (default). This forces tests to execute
669 # after the build, including when running build related tasks like install, upload and release.
670 #
671 # Set to false to not run any tests. Set to :all to run all tests, ignoring failures.
672 #
673 # This option is set from the environment variable 'test', so you can also do:
674
675 # Returns the test option (environment variable TEST). Possible values are:
676 # * :false -- Do not run any tests (also accepts 'no' and 'skip').
677 # * :true -- Run all tests, stop on failure (default if not set).
678 # * :all -- Run all tests, ignore failures.
679 def test
680 case value = ENV['TEST'] || ENV['test']
681 when /^(no|off|false|skip)$/i
682 false
683 when /^all$/i
684 :all
685 when /^(yes|on|true)$/i, nil
686 true
687 else
688 warn "Expecting the environment variable test to be 'no' or 'all', not sure what to do with #{value}, so I'm just going to run all the tests and stop at failure."
689 true
690 end
691 end
692
693 # Sets the test option (environment variable TEST). Possible values are true, false or :all.
694 #
695 # You can also set this from the environment variable, e.g.:
696 #
697 # buildr # With tests
698 # buildr test=no # Without tests
699 # buildr test=all # Ignore failures
700 # set TEST=no
701 # buildr # Without tests
702 def test=(flag)
703 ENV['test'] = nil
704 ENV['TEST'] = flag.to_s
705 end
706
707 end
708
709 Buildr.help << <<-HELP
710 To run a full build without running any tests:
711 buildr test=no
712 To run specific test:
713 buildr test:MyTest
714 To run integration tests:
715 buildr integration
716 HELP
717
718 end
719
720
721 class Buildr::Project
722 include Buildr::Test
723 end
Generated using the rcov code coverage analysis tool for Ruby
version 0.8.2.1.