Elvas Tower: Debrief Evaluation - Elvas Tower

Jump to content

  • 10 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

Debrief Evaluation

#1 User is offline   ATW 

  • Engineer
  • Group: Status: Active Member
  • Posts: 526
  • Joined: 07-January 13
  • Gender:Male
  • Simulator:MSTS Open Rails
  • Country:

Posted 03 December 2015 - 09:21 PM

With all the talk about activities, physics and realism and great progress in ORTS development being made I would like to ask and talk about the main topic...

Since we have already represented curve speed based on wag/eng settings an yet more to come even with other current features that can be used, have we considered activity completion/failed evaluation de briefings like MSTS instead of just completing quitting exiting with no feedback, scores and ratings as well as opportunity to save them in a file like MSTS for reviews? Even though MSTS never gave any scores or star ratings you will know what I mean for those that play Railworks getting scored based on your speed, handling, station timings, performance etc. With MSTS you get told if you passed or failed something like exceeded durability levels a number of times which I like to represent in ORTS based on wag/eng physics handling curve speeds.

Yes we have the event like recorder of logging train speed in the options but what about a basic MSTS Railworks alike one that gives you feedback, time of event and ratings? I knottice the current ORTS Evaluation logging does not work even when checked as I can't seem to find the file where it use to be anymore. Example's bellow in detail gives you understanding of what I look for asking and would like others thoughts in this.

1. Time, number of times and place of a high critical event triggered by the following: Broken Coupler; Emergency Brake Application Automatic, Manuel or Mechanically but shouldn't be critical if train was already stopped where event got triggered unless it moved the train; Station Departure movement while loading unloading passengers; Failed to sound horn or bell for road crossings or stations; Passed STOP indication signal with no permission; Ran Through misaligned switch or buffer "Red Block line in Track Monitor"; Train overturned; Track Speed Limit exceeded severely (Track monitor track in Red); etc

1b. Not currently implemented yet to take part from scratch but a part of a high critical event: Train was jack knifed overturned; Train was string lined over accelerated; Train coupled too fast; etc



2. Time, number of times and place of a low critical event triggered by the following: You missed some station stops; You were late arriving stations; You exceeded Durability levels; You missed a pickup/drop off; Train was not assembled correctly as work order; You exceeded the speed limit by not too much (Track Monitor track in orange); etc

3. Opposite of the mentions above is positive feedback as well as feedback in triggered texts like MSTS shows based on location events with success with bonus rating points.


Any other thoughts or opinions for some good ORTS Evaluation triggers and even a possible fuel conservation telling how much total fuel we used?

#2 User is offline   mirekkr 

  • Hostler
  • Group: Status: Active Member
  • Posts: 91
  • Joined: 18-January 15
  • Simulator:ORTS, MSTS
  • Country:

Posted 04 December 2015 - 06:46 AM

I fully agree to this.
If we prefer realism we must also deal with failures some way.
Thanks for this thread, ATW.

Regards,
Mirek

#3 User is offline   James Ross 

  • Open Rails Developer
  • Group: Status: Elite Member
  • Posts: 5,181
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 05 December 2015 - 04:38 AM

We have a blueprint for doing activity evaluations but nobody's really taken the time to spec it out.

View PostATW, on 03 December 2015 - 09:21 PM, said:

Yes we have the event like recorder of logging train speed in the options but what about a basic MSTS Railworks alike one that gives you feedback, time of event and ratings? I knottice the current ORTS Evaluation logging does not work even when checked as I can't seem to find the file where it use to be anymore. Example's bellow in detail gives you understanding of what I look for asking and would like others thoughts in this.

1. Time, number of times and place of a high critical event triggered by the following: Broken Coupler; Emergency Brake Application Automatic, Manuel or Mechanically but shouldn't be critical if train was already stopped where event got triggered unless it moved the train; Station Departure movement while loading unloading passengers; Failed to sound horn or bell for road crossings or stations; Passed STOP indication signal with no permission; Ran Through misaligned switch or buffer "Red Block line in Track Monitor"; Train overturned; Track Speed Limit exceeded severely (Track monitor track in Red); etc

1b. Not currently implemented yet to take part from scratch but a part of a high critical event: Train was jack knifed overturned; Train was string lined over accelerated; Train coupled too fast; etc



2. Time, number of times and place of a low critical event triggered by the following: You missed some station stops; You were late arriving stations; You exceeded Durability levels; You missed a pickup/drop off; Train was not assembled correctly as work order; You exceeded the speed limit by not too much (Track Monitor track in orange); etc

3. Opposite of the mentions above is positive feedback as well as feedback in triggered texts like MSTS shows based on location events with success with bonus rating points.

These all seem reasonable although some (e.g. "Failed to sound horn or bell for road crossings or stations") might not have the necessary data to determine whether you did the right thing or not (e.g. I'm not sure if there's any data indicating whether you need to sound your horn or not). For most things though, we can do that determination. :)

If you'd like to spec out exactly what things contribute "good" and "bad" towards an evaluation, please feel free to do so and we can let someone start implementing bits of it.

FWIW, most of the things should work exactly the same in activity and timetable mode and I'd like this to be supported in both.

#4 User is offline   ATW 

  • Engineer
  • Group: Status: Active Member
  • Posts: 526
  • Joined: 07-January 13
  • Gender:Male
  • Simulator:MSTS Open Rails
  • Country:

Posted 05 December 2015 - 12:01 PM

Yea the horn bell one may be not too necessary... I just brought it up in connection of some crossing crash probability settings making MSTS activity kick you out where you didn't sound horn it's a game over exit.

I will come up with a good spec list soon.

#5 User is offline   ATW 

  • Engineer
  • Group: Status: Active Member
  • Posts: 526
  • Joined: 07-January 13
  • Gender:Male
  • Simulator:MSTS Open Rails
  • Country:

Posted 06 December 2015 - 01:10 AM

Evaluation Spec ORTS

Ratings & STARS

No special Graphic is needed if not necessary as we can start an use numbers instead as the following view as well as points all in writing:

1 STAR= 20 Points
2 STAR= 40 Points
3 STAR= 60 Points
4 STAR= 80 Points
5 STAR= 100 Points

Points to meet STAR rating can be used as both gaining and losing method. From a point of view depending on what needs to be evaluated and what the consequence in lost of points is of that particular thing needing evaluation, you may already start a run at having 5 Stars regardless of how much time of your run or assignment is where if you do something in explore you have nothing to lose unless you do something wrong to lose points. But if you don't complete a run assignment, work order or whole activity, you lose points for what ever was failed. Bonus points may also be used, used as needed to make up from something like the example of time... early arrival for gain and late arrival for lose. Overall total points leads to your average score of your skills in that run whether 0 Star Rookie, 1 Star Poor, 2 Star Bad, 3 Star Good, 4 Star Great, 5 Star Pro or overrated beyond 5 star Expert counted from additional bonus making points. Examples can be added up, subtracted, calculated and seen in the rating tabs of what outcomes of a certain thing being evaluated are and what points are given.

That's the first half starting with ratings and stars info. Lets get to the main point of evaluation outcomes and examples to follow of my idea with or without scoring needed as some may be necessary or not?

SPEED Evaluation:

SPEED Rating Deduction Table: -0.2 Points per Second Over speeding Track Monitor Orange, -1.5 Points per Second Over-speeding Track Monitor Red

Example of Bad Feedback Notes: Speed was not maintained well because you exceeded the speed limit 2 times for where you were going a max of 5MPH over a speed limit for 2 Minutes 14 Seconds.
1. Between Siding markers Woodford and Walong a Max Speed of 25MPH for 0 Minutes 10 Seconds for the Speed limit exceeded was 23MPH at 11:03:34. -2 points
2. Between Siding markers East Bealville and Cliff a Max Speed of 28MPH for 2 Minutes 14 Seconds for the Speed limit exceeded was 23MPH at 11:55:22. -26.8 points
Overall Rating Total= 1 STAR -28.8 Points deduction from 5 STAR 100= 71.2 Points 3 STAR rating for SPEED.


Example of Good Feedback Note: Speed was maintained very well an safely for you did not exceed the speed limits for track, signals, train and other possible restrictions on your journey which is excellent.
Overall Rating Total= 5 STAR rating for SPEED


Freight Durability/Passenger Comfort Evaluation:

DURABILITY Rating Deduction Table: -3 Points per out comes of ENG/WAG curve speeds exceeded, -7 Points per Hose Breaks, -10 Points per Coupler Knuckle Breaks, -20 Points per Overturned, String lined or Jackknifed ENG/WAG,

Example of Bad Feedback Notes: Freight Durability limits were exceeded 5 times for there could either have been damage done to loaded freight commodity, car body, equipment wear or all and worse.
1. Between Siding markers Cable X Over and Tehachapie a Hose snapped off on CONSISTS number 40 of your train at 10:30:07. -7 Points
2. Between Siding markers Ilmon and Bena CONSISTS number 29 of your train durability speed at curve was exceeded at 12:44:44. -3 Points
3. Between Siding markers Sandcut and Edison CONSISTS numbers 10, 12, 16 on your train were Jackknifed due to high run in train forces where bunch durability limits were exceeded by poor handling or train make up at 12:58:00. -60 Points
4. Between Siding markers UP Yard 3 and East Bakersfield CONSISTS number 11 on your train Coupler broke due to coupler limits being exceeded at 14:04:04. -10 Points
5. Between Siding markers UP Yard 3 and East Bakersfield CONSISTS number 6, 7, 8 on your train String lined at curve where acceleration durability limits were exceeded at 14:04:13. -60 Points
Overall Rating Total= 5 STAR -140 Points deduction from 5 STAR= -40 points 0 STAR rating for Durability

Example of Good Feedback Note: Freight and Passenger Comfort durability levels were well maintained as you handled the comfort and durability of your train smoothly with no damage.
Overall Rating Total= 5 STAR rating for DURABILITY & PASSENGER COMFORT.

EMERGENCY/PENALTY Actions Evaluation:

EMERGENCY-PENALTY Rating Deduction Table: -2 Points per Penalty Full Brake applications under 5MPH, -5 Points per Emergency applications while STOPPED, -20 Points per Emergency applications MOVING, -35 Points per Alerter Penalty applications above 10MPH.

Example of Bad Feedback Notes: There was 3 Critical operational errors counted.
1. Between Siding 901 and CP Alameda 2 a Emergency application was applied at a Max Speed of 18MPH at 10:51:06. -20 Points
2. Between Siding 901 and CP Alameda 2 a Emergency application was applied at a Max Speed of 0MPH at 10:58:43. -5 Points
3. Between Station Montebello 2 and Maple 2 a Alerter Penalty application was applied at a Max speed of 27MPH at 13:44:09. -35 Points
Overall Rating Total= 3 STAR -60 Point deduction from 5 STAR= 40 points 2 STAR rating for EMERGENCY and PENALTY actions.

Example of Good Feedback Note: Actions free of any Emergency Penalty incidents were maintained as there was no operational errors.
Overall Rating Total= 5 STAR rating for EMERGENCY PENALTY action.

STATION Arrival, Departure, Passing Evaluation:

STATION Rating Deduction Table: -0.2 Points per Second delayed, -20 Points per Stations missed from stopping, -20 Points per minute delayed from passing stations (averagely freight trains with time strict priority if set by activity maker where trains do not need to stop), -80 Points per outcomes of departing Passenger Stations without permission to proceed while loading/unloading.

STATION Rating Increase/Bonus Table: +0.2 Points per Second early or on time, +10 points Per Safe departures with permission, +20 Points Per Minute early on time from passing stations (averagely freight trains with time strict priority if set by activity maker where trains do not need to stop). Speeding points are deducted from SPEED Evaluation even if one try's to make up for time bonus for stations whether user chooses safety or priority 1st.

Example of Bad Feedback Notes: There were some station arrival/departures issues. You were late arriving at 1 station. You were late passing a station. You commuted a safety violation error.
1. You were 4 minutes 4 seconds late arriving at Montebello Platform 2 arriving at 11:04:04. -56 points
2. You departed Montebello Platform 2 without permission for there were passengers loading an unloading an could of been injured due to you moving. -80 points
3. You were 1 minute late passing Pedley Station passing at 11:56:22. -20 points
Overall Rating Total= 5 STAR -156 Points deduction from 5 STAR 100= -56 Points 0 STAR rating for STATION Arrival/Departure/Passing.


Example of Good Feedback Note: All Station arrival goals were met and you are recognized. You did not miss any station stops. You didn't pass or arrive at any stations late and kept your crew and passengers safe by not proceeding without permission during loading/unloading and deserve these bonuses.
1. You were 3 minutes 13 seconds early arriving at Pomona Platform 2 arriving at 11:57:47. +38.6 make up or bonus points
2. You were 5 minutes early passing Pedley arriving at 12:32:47. +100 make up or bonus points
3. You were 15 seconds early arriving at Riverside Platform 1 arriving at 12:44:45. +3 make up or bonus points
4. With the 3 station Stops start to end you did not commit any platform safety violations and are awarded for your safety of not departing without permission. +30 make up or bonus points
Overall Rating Total= 5 STAR rating and beyond with bonus 171.6 points= 8 STARS



Still more Evaluation tab spec ideas to put together in text but that's the first I have written down starting basically from train handling, controlling speeds being alert and operational error actions in braking. Next post or post edit I should try things related to station arrival/departures, work orders, event triggered evaluation texts then afterwords other bonus related stuff then put together into full evaluation example pics.

Edit: Station performance is added on 1/20/16

#6 User is offline   dforrest 

  • Engineer
  • Group: Status: Active Member
  • Posts: 718
  • Joined: 12-January 12
  • Gender:Male
  • Location:St. Vincent (formally UK)
  • Simulator:MSTS, Open Rails
  • Country:

Posted 06 December 2015 - 05:44 AM

ATW, I support you fully on this endeavour.

I will consider further particular additions which which I thing are necessary and post these. An initial comment is that there should be a loss of points when water is taken on away from a filling point.

#7 User is offline   timothyskinner 

  • Conductor
  • Group: Status: Active Member
  • Posts: 274
  • Joined: 18-September 13
  • Gender:Male
  • Simulator:msts
  • Country:

Posted 06 December 2015 - 06:16 AM

View Postdforrest, on 06 December 2015 - 05:44 AM, said:

I will consider further particular additions which which I thing are necessary and post these. An initial comment is that there should be a loss of points when water is taken on away from a filling point.

From my experience in the Virtuals time taken overall is not a good enough measure of performance.

That is, liberal use of the Save then later Load Save enables a player to correct mistakes and go on to achieve a 5-star performance. An evaluation not allowing any Saves gives a better measure of performance. A means of disallowing any practice is much more difficult to program in.

And, of course, in MSTS at least, allowing players to change certain parameters enables a better performance. For example, a switching activity written for heavy rain or snow at 2am is made far easier when changed to clear visibility at 11am...

#8 User is offline   dforrest 

  • Engineer
  • Group: Status: Active Member
  • Posts: 718
  • Joined: 12-January 12
  • Gender:Male
  • Location:St. Vincent (formally UK)
  • Simulator:MSTS, Open Rails
  • Country:

Posted 03 January 2016 - 03:01 AM

Anything more on this?

#9 User is offline   Hamza97 

  • Engineer
  • Group: Status: Active Member
  • Posts: 547
  • Joined: 01-March 15
  • Gender:Male
  • Simulator:Open Rails
  • Country:

Posted 03 January 2016 - 04:19 AM

Can we have some additional evaluating parameter...??

Like for e.g. in diesels we can have a parameter for litres of diesel fuel used, if more is used less points and if less is used more points can be awarded.

Similarly for electrics, we can have more or less points depending on how Much "juice" we drew from catanery.

For steam it would be amount of water and coal used.

#10 User is offline   ATW 

  • Engineer
  • Group: Status: Active Member
  • Posts: 526
  • Joined: 07-January 13
  • Gender:Male
  • Simulator:MSTS Open Rails
  • Country:

Posted 20 January 2016 - 12:30 AM

Post 5 is updated with evaluation views of STATION Arrivals, Departures, Passings.

As for fuel bonuses I was thinking the same before as there is a MSTS con thing related to train efficiency that even messes with AI performance in MSTS an not in ORTS yet but need to put a evaluation spec view together when I get to bonuses. One bonus loss I can think of related to points loss for one that wants to makeup from a save without completing in one straight piece is one where if someone saves an exits an resumes from save... points are deducted because it's like cheating being the same as pausing an auto pilot a number of times where your not a real engineer completing challenges as in real life an better have un disturbed time or pull your train over if act an dispatch permit you not to lose points based on time delay.

Anyone know how to get the current Event A like Recorder Evaluation to display in a text or doc file properly in organized tabs like before an get the recorded lines to record every 5 seconds or another chosen time in order to save memory size of the file?

  • 10 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users