Opened 2 years ago

Closed 21 months ago

Last modified 21 months ago

#14660 closed task (fixed)

Improve +RTS -t --machine-readable

Reported by: duog Owned by:
Priority: normal Milestone: 8.6.1
Component: Runtime System Version: 8.3
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D4303
Wiki Page:

Description

The output of +RTS -t --machine-readable is a shown Haskell list, so we can easily add fields. Let's add everything shown by +RTS -s . I've posted a rough patch that does that at Phab:D4303.

If this is a good idea, I'd like to also add the following:

  • a data structure and parsing function to GHC.Stats in base.
  • The ghc version as a field in the --machine-readable output
  • The rts ways as a field in the --machine-readable output

Change History (7)

comment:1 Changed 2 years ago by mpickering

It could just be output as a JSON record rather than relying on having to use Haskell tooling to parse it.

comment:2 Changed 2 years ago by duog

Using a JSON record would break existing tooling, which is a little on the nose since this output has been documented as being forwards compatible.

It would be good to have a nice structured record for the per-generations stats though.

comment:3 Changed 2 years ago by duog

Differential Rev(s): Phab:D4303

comment:4 Changed 21 months ago by Ben Gamari <ben@…>

In 2d4bda2e/ghc:

rts, base: Refactor stats.c to improve --machine-readable report

There should be no change in the output of the '+RTS -s' (summary)
report, or the 'RTS -t' (one-line) report.

All data shown in the summary report is now shown in the machine
readable report.

All data in RTSStats is now shown in the machine readable report.

init times are added to RTSStats and added to GHC.Stats.

Example of the new output:
```
 [("bytes allocated", "375016384")
 ,("num_GCs", "113")
 ,("average_bytes_used", "148348")
 ,("max_bytes_used", "206552")
 ,("num_byte_usage_samples", "2")
 ,("peak_megabytes_allocated", "6")
 ,("init_cpu_seconds", "0.001642")
 ,("init_wall_seconds", "0.001027")
 ,("mut_cpu_seconds", "3.020166")
 ,("mut_wall_seconds", "0.757244")
 ,("GC_cpu_seconds", "0.037750")
 ,("GC_wall_seconds", "0.009569")
 ,("exit_cpu_seconds", "0.000890")
 ,("exit_wall_seconds", "0.002551")
 ,("total_cpu_seconds", "3.060452")
 ,("total_wall_seconds", "0.770395")
 ,("major_gcs", "2")
 ,("allocated_bytes", "375016384")
 ,("max_live_bytes", "206552")
 ,("max_large_objects_bytes", "159344")
 ,("max_compact_bytes", "0")
 ,("max_slop_bytes", "59688")
 ,("max_mem_in_use_bytes", "6291456")
 ,("cumulative_live_bytes", "296696")
 ,("copied_bytes", "541024")
 ,("par_copied_bytes", "493976")
 ,("cumulative_par_max_copied_bytes", "104104")
 ,("cumulative_par_balanced_copied_bytes", "274456")
 ,("fragmentation_bytes", "2112")
 ,("alloc_rate", "124170795")
 ,("productivity_cpu_percent", "0.986838")
 ,("productivity_wall_percent", "0.982935")
 ,("bound_task_count", "1")
 ,("sparks_count", "5836258")
 ,("sparks_converted", "237")
 ,("sparks_overflowed", "1990408")
 ,("sparks_dud ", "0")
 ,("sparks_gcd", "3455553")
 ,("sparks_fizzled", "390060")
 ,("work_balance", "0.555606")
 ,("n_capabilities", "4")
 ,("task_count", "10")
 ,("peak_worker_count", "9")
 ,("worker_count", "9")
 ,("gc_alloc_block_sync_spin", "162")
 ,("gc_alloc_block_sync_yield", "0")
 ,("gc_alloc_block_sync_spin", "162")
 ,("gc_spin_spin", "18840855")
 ,("gc_spin_yield", "10355")
 ,("mut_spin_spin", "70331392")
 ,("mut_spin_yield", "61700")
 ,("waitForGcThreads_spin", "241")
 ,("waitForGcThreads_yield", "2797")
 ,("whitehole_gc_spin", "0")
 ,("whitehole_lockClosure_spin", "0")
 ,("whitehole_lockClosure_yield", "0")
 ,("whitehole_executeMessage_spin", "0")
 ,("whitehole_threadPaused_spin", "0")
 ,("any_work", "1667")
 ,("no_work", "1662")
 ,("scav_find_work", "1026")
 ,("gen_0_collections", "111")
 ,("gen_0_par_collections", "111")
 ,("gen_0_cpu_seconds", "0.036126")
 ,("gen_0_wall_seconds", "0.036126")
 ,("gen_0_max_pause_seconds", "0.036126")
 ,("gen_0_avg_pause_seconds", "0.000081")
 ,("gen_0_sync_spin", "21")
 ,("gen_0_sync_yield", "0")
 ,("gen_1_collections", "2")
 ,("gen_1_par_collections", "1")
 ,("gen_1_cpu_seconds", "0.001624")
 ,("gen_1_wall_seconds", "0.001624")
 ,("gen_1_max_pause_seconds", "0.001624")
 ,("gen_1_avg_pause_seconds", "0.000272")
 ,("gen_1_sync_spin", "3")
 ,("gen_1_sync_yield", "0")
 ]
```

Test Plan: Ensure that one-line and summary reports are unchanged.

Reviewers: bgamari, erikd, simonmar, hvr

Reviewed By: simonmar

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #14660

Differential Revision: https://phabricator.haskell.org/D4303

comment:5 Changed 21 months ago by bgamari

Resolution: fixed
Status: newclosed

comment:6 Changed 21 months ago by bgamari

Milestone: 8.6.1

comment:7 Changed 21 months ago by Ben Gamari <ben@…>

In f0b258b/ghc:

rts, base: Refactor stats.c to improve --machine-readable report

There should be no change in the output of the '+RTS -s' (summary)
report, or
the 'RTS -t' (one-line) report.

All data shown in the summary report is now shown in the machine
readable
report.

All data in RTSStats is now shown in the machine readable report.

init times are added to RTSStats and added to GHC.Stats.

Example of the new output:
```
 [("bytes allocated", "375016384")
 ,("num_GCs", "113")
 ,("average_bytes_used", "148348")
 ,("max_bytes_used", "206552")
 ,("num_byte_usage_samples", "2")
 ,("peak_megabytes_allocated", "6")
 ,("init_cpu_seconds", "0.001642")
 ,("init_wall_seconds", "0.001027")
 ,("mut_cpu_seconds", "3.020166")
 ,("mut_wall_seconds", "0.757244")
 ,("GC_cpu_seconds", "0.037750")
 ,("GC_wall_seconds", "0.009569")
 ,("exit_cpu_seconds", "0.000890")
 ,("exit_wall_seconds", "0.002551")
 ,("total_cpu_seconds", "3.060452")
 ,("total_wall_seconds", "0.770395")
 ,("major_gcs", "2")
 ,("allocated_bytes", "375016384")
 ,("max_live_bytes", "206552")
 ,("max_large_objects_bytes", "159344")
 ,("max_compact_bytes", "0")
 ,("max_slop_bytes", "59688")
 ,("max_mem_in_use_bytes", "6291456")
 ,("cumulative_live_bytes", "296696")
 ,("copied_bytes", "541024")
 ,("par_copied_bytes", "493976")
 ,("cumulative_par_max_copied_bytes", "104104")
 ,("cumulative_par_balanced_copied_bytes", "274456")
 ,("fragmentation_bytes", "2112")
 ,("alloc_rate", "124170795")
 ,("productivity_cpu_percent", "0.986838")
 ,("productivity_wall_percent", "0.982935")
 ,("bound_task_count", "1")
 ,("sparks_count", "5836258")
 ,("sparks_converted", "237")
 ,("sparks_overflowed", "1990408")
 ,("sparks_dud ", "0")
 ,("sparks_gcd", "3455553")
 ,("sparks_fizzled", "390060")
 ,("work_balance", "0.555606")
 ,("n_capabilities", "4")
 ,("task_count", "10")
 ,("peak_worker_count", "9")
 ,("worker_count", "9")
 ,("gc_alloc_block_sync_spin", "162")
 ,("gc_alloc_block_sync_yield", "0")
 ,("gc_alloc_block_sync_spin", "162")
 ,("gc_spin_spin", "18840855")
 ,("gc_spin_yield", "10355")
 ,("mut_spin_spin", "70331392")
 ,("mut_spin_yield", "61700")
 ,("waitForGcThreads_spin", "241")
 ,("waitForGcThreads_yield", "2797")
 ,("whitehole_gc_spin", "0")
 ,("whitehole_lockClosure_spin", "0")
 ,("whitehole_lockClosure_yield", "0")
 ,("whitehole_executeMessage_spin", "0")
 ,("whitehole_threadPaused_spin", "0")
 ,("any_work", "1667")
 ,("no_work", "1662")
 ,("scav_find_work", "1026")
 ,("gen_0_collections", "111")
 ,("gen_0_par_collections", "111")
 ,("gen_0_cpu_seconds", "0.036126")
 ,("gen_0_wall_seconds", "0.036126")
 ,("gen_0_max_pause_seconds", "0.036126")
 ,("gen_0_avg_pause_seconds", "0.000081")
 ,("gen_0_sync_spin", "21")
 ,("gen_0_sync_yield", "0")
 ,("gen_1_collections", "2")
 ,("gen_1_par_collections", "1")
 ,("gen_1_cpu_seconds", "0.001624")
 ,("gen_1_wall_seconds", "0.001624")
 ,("gen_1_max_pause_seconds", "0.001624")
 ,("gen_1_avg_pause_seconds", "0.000272")
 ,("gen_1_sync_spin", "3")
 ,("gen_1_sync_yield", "0")
 ]
```

Test Plan: Ensure that one-line and summary reports are unchanged.

Reviewers: erikd, simonmar, hvr

Subscribers: duog, carter, thomie, rwbarton

GHC Trac Issues: #14660

Differential Revision: https://phabricator.haskell.org/D4529
Note: See TracTickets for help on using tickets.