Gaming Machine Reacting Abnormally at the Same Time Every Day What Causes It
Of all the abnormal machine behaviors that operators report, the daily timed behavior is the most diagnostically useful. When a machine does something unusual at the same time every day — reboots at 3 AM, shows an error code at 9 AM, acts sluggish at 2 PM — the timing is the diagnostic clue. The timing narrows the possible causes to three categories: scheduled events (the machine has a timer that triggers the behavior), environmental events (an external condition that occurs at that time triggers the behavior), and scheduled attacks (the attacker targets that time for reasons known to them). A bus-monitoring device records the exact timing and the bus events surrounding the behavior, enabling the diagnosis of which category is responsible and what specific mechanism within that category. This article explains how to diagnose daily timed abnormal behaviors using bus data.
The Diagnostic Power of Exact Timing
The exact timing of the abnormal behavior is the most important diagnostic information. “Around 3 AM” is not sufficient. “At 3:07 AM, plus or minus 2 minutes” is diagnostic. The precision and the variation in the timing reveal the mechanism. A behavior that occurs at exactly the same time every day — for example, at 3:00:00 AM exactly, every day — indicates a scheduled timer. The machine firmware or operating system has a timer that triggers the behavior at that precise time. The timer clock is synchronized to the machine real-time clock, which maintains accurate time. The timer-based behavior is typically a maintenance task — a self-test, a log cleanup, or a data backup — that the manufacturer programmed to run at a specific time. The behavior may appear abnormal to the operator because the operator does not know about the scheduled maintenance task. The bus-monitoring device records the behavior timing and the bus events. Comparison with the machine documentation or consultation with the manufacturer confirms that the behavior is a scheduled maintenance task. No corrective action is needed. The operator should document the behavior for future reference.
A behavior that occurs at approximately the same time but with significant variation — for example, at 3:07 AM on Monday, 3:12 AM on Tuesday, and 3:03 AM on Wednesday — indicates an external trigger that is approximately synchronized to the time of day. The external trigger may be: the building power grid switching to a different transformer (causes a voltage transient that the machine power supply does not handle well), the street lights turning on (causes RF interference that couples onto the machine bus), or the neighbor business opening or closing (causes vibration or power load changes). The variation in timing is because the external trigger is approximate, not precise. The bus-monitoring device records the behavior timing and the bus events. The bus events immediately before the behavior provide the trigger signature — for example, a voltage transient on the power supply monitor line or a signal burst on the bus lines. The trigger signature identifies the external cause. The corrective action is to isolate the machine from the external trigger — for example, by installing a line conditioner to filter power transients, by adding RF shielding to block external interference, or by relocating the machine away from the vibration source.
A behavior that occurs at irregular times — for example, at 3:07 AM on Monday, at 9:15 PM on Wednesday, and at 4:22 PM on Friday — indicates a human-triggered event, not a timer or an environmental trigger. The human trigger is the attacker activating the control device according to their own schedule. The bus-monitoring device records the behavior timing and the bus events. The bus events include the control command that triggered the behavior. The command signature identifies the attack method. The command timing identifies the attacker schedule. The investigation follows the external control diagnostic checklist described in the previous article.
Environmental Triggers: Daily Events That Affect Machine Behavior
The daily environmental cycle creates numerous potential triggers for machine behavior. The most common triggers are: air conditioning startup (which causes a voltage drop on the power line), street lights activation (which causes electromagnetic interference), clean shift startup (when the cleaning staff starts their vacuum cleaners, causing vibration and power noise), and the building electrical system peak load shift (when the building switches between different transformer configurations). Each of these triggers occurs at approximately the same time every day — within a window of 10 to 30 minutes — and can affect machine behavior if the machine is electrically or mechanically coupled to the trigger source.
Diagnosing environmental triggers requires comparing the machine behavior timing with the building event timing. The building event timing is available from the building management system logs, the cleaning staff schedule, or the power company load schedule. If the machine behavior timing correlates with one of these building events, the building event is the likely trigger. The bus-monitoring device confirms the trigger by recording the bus events at the trigger time. The device log shows a power anomaly event (if the trigger is electrical), a signal anomaly event (if the trigger is electromagnetic), or a sensor anomaly event (if the trigger is vibration). The anomaly type confirms the trigger mechanism. The corrective action is to isolate the machine from the trigger: install a line conditioner for electrical triggers, install RF shielding for electromagnetic triggers, or install vibration isolation mounts for vibration triggers.
The isolation solution is preferable to trying to eliminate the building event. You cannot ask the building to stop turning on the air conditioning to prevent a machine behavioral anomaly. The building event will continue. The machine must be made resilient to it. The bus-monitoring device verifies the resilience by showing that the anomaly events disappear after the isolation is installed. The verification confirms that the isolation solution was effective. If the anomaly events continue, the isolation was insufficient, and additional measures are needed. The device provides the feedback loop that ensures the solution works.
Scheduled Attacks: The Attacker Timer
Some attackers use a timer to schedule their attacks. The timer may be an electronic device that the attacker programs to activate at a specific time — for example, an RF transmitter with a timer that activates at 3 AM when the attacker assumes no one is watching. The timer may also be a software timer in malware that the attacker has installed on the machine — for example, malware that activates at 2 PM every day to extract credits during the afternoon peak. The timer-based attack has the advantage for the attacker that they do not need to be present during the attack. The attack occurs automatically. The attacker collects the extracted credits at their convenience — for example, by visiting the venue on a later day and playing the machine to collect the accumulated credits.
Timer-based attacks are detected by the bus-monitoring device as anomalous events that occur at precisely timed intervals. The device log shows a credit injection event at exactly 2:00 PM on Monday, another at exactly the same time on Tuesday, and another on Wednesday. The precise timing indicates a timer. The bus events at the attack times show the attack signal characteristics — the signal type (credit injection, payout command, or configuration write), the signal source (which bus line), and the signal format. The signal characteristics identify the attack device type. The timing identifies the attack schedule. The investigation focuses on finding the timer device on the machine or in the venue vicinity.
The timer device search is conducted at a time when the timer is not scheduled to activate — to avoid alerting the attacker. The search includes: checking the diagnostic port for a timer device, checking the machine interior for a timer circuit board, and scanning the venue for RF transmitters that may have a timer module. The timer device may also be controlled remotely — the attacker sends a wake-up signal to activate the timer, and the timer then executes the attack. The remote wake-up adds complexity to the investigation. The wake-up signal may be a different frequency or a different protocol than the attack signal. The bus-monitoring device records both the wake-up signal and the attack signal, providing the complete picture of the timer-based attack sequence.
Responding to Scheduled Attacks
The response to a scheduled attack has a unique opportunity that the response to an unscheduled attack does not have: you know when the next attack will occur. The timer schedule is predictable. You can position security resources to intercept the attack. The interception strategy is: review the bus log to determine the exact attack time, schedule a staff member to be present at the machine at that time, and have the staff member observe the machine behavior and the surrounding area. If the attack involves an external RF signal, the staff member may be able to locate the transmitter by following the signal strength. If the attack involves an internal timer device, the staff member can open the machine immediately after the attack activates and locate the device. The scheduled interception is the most effective countermeasure for timer-based attacks because it catches the attack in the act and provides the evidence to identify the attacker or the device.
The scheduled interception should be conducted discreetly. The staff member should appear to be performing routine maintenance, not surveillance. If the attacker is present in the venue, the attacker will be watching the machine and may notice if a staff member is standing next to the machine at the exact attack time. The attacker may abort the attack if they suspect they are being watched. The discretion maximizes the probability of catching the attacker. The staff member should have a phone or a camera to record the machine behavior and the surrounding area. The recording provides the evidence for the investigation and for any legal action.
After the interception, the timer device is removed, and the machine operation is verified with the bus-monitoring device. The device log should show no further timer-triggered events for at least 7 days. The 7-day verification period ensures that all timer devices have been removed — including any secondary timer devices that may have been installed as backups. The verification period is longer than for other types of attacks because the timer cycle may be longer than 24 hours — the timer may activate on specific days of the week rather than every day. The 7-day period covers all possible days of the week, ensuring that no timer remains active.
Frequently Asked Questions
Can the bus monitor distinguish between a scheduled maintenance task and a scheduled attack? Yes, by the bus events that accompany the behavior. A scheduled maintenance task produces bus events that are within the normal baseline — for example, a self-test that reads the machine status registers. A scheduled attack produces bus events that are outside the normal baseline — for example, a credit injection that writes to the counter registers. The bus events are different because the activities are different. The device classifies the events based on the baseline and reports whether they are normal (maintenance) or anomalous (attack). The operator can also consult the machine documentation to determine which maintenance tasks are scheduled and when they occur. The documentation provides the expected maintenance schedule. The device log provides the actual behavior. Comparison between expected and actual identifies any behavior that is not part of the documented maintenance schedule.
What if the daily timed behavior is caused by a player who visits the venue at the same time every day? The player behavior would appear in the bus log as normal play activity — button presses with human timing variation, credit events from coin insertion, and standard game play signals. The player behavior would not produce anomalous signals because the player is playing the machine normally. The machine behavior would be different from other times of day because the play volume is different — if the player is a high-volume player, the machine would process more transactions during the player visit. The increased transaction volume is visible in the bus log as a higher density of normal events. The higher density is legitimate — it is caused by a high-volume player, not by an attack. The bus log distinguishes between high-volume legitimate play (normal event types, high density) and attack (anomalous event types, abnormal density). The distinction prevents the operator from mistakenly investigating a loyal, high-spending customer as if they were an attacker.
Can the bus monitor help me determine whether to disable a scheduled maintenance task that I think is causing abnormal behavior? Yes. The device log records the maintenance task events and their timing. If the maintenance task is causing a brief period of abnormal behavior — for example, the self-test temporarily reduces the machine responsiveness — the device log shows the duration of the abnormal period. The operator can decide whether the abnormal period is acceptable. A 30-second period of reduced responsiveness every night at 3 AM is acceptable for most venues because no players are present. A 5-minute period of reduced responsiveness during business hours — because the maintenance task was incorrectly scheduled — is not acceptable and should be rescheduled. The device log enables the operator to make an informed decision about the maintenance schedule based on the actual impact on machine operation.