Model Accuracy & Validation
A forecast you do not measure is an opinion with decimals - accuracy tracking is what makes prediction a discipline.
query_statsThe Accuracy Metrics
A forecast is a number; accuracy is the verdict on it. Six metrics cover the ground - each answers a different question about how wrong the model is, and in which direction:
MAPE
Mean absolute percentage error - the industry default because it is intuitive to read: 8% error means 8% off, on average.
MAE & RMSE
Absolute and root-mean-square error, measured in rooms rather than percent - RMSE punishes big misses harder than small ones.
Forecast Bias
Systematic over- or under-forecasting - the most dangerous error because it compounds across every decision it feeds.
Accuracy by Segment
Blended accuracy hides segment disasters - a healthy overall score can conceal a badly wrong weekday-business forecast.
Hit Rate
The share of forecasts landing within tolerance - a simple pass/fail view of reliability that leadership reads at a glance.
Value Score
Accuracy weighted by the revenue impact of the decision it fed - error on high-stakes dates counts far more.
fact_checkBacktesting & Validation
The rule is simple: never judge a model on the data it was trained on. Hold out the most recent periods, forecast them as if blind, and score the results honestly against what actually happened.
One lucky month proves nothing. Walk-forward testing repeats the exercise across many dates - train, forecast, step forward, repeat - so the score reflects real conditions: high seasons and low, events and holidays, missing data and late pickups.
Validation separates a model that performed once from a model that performs. Rerun it whenever the model, the data or the market changes - not just at launch.
psychologyOverfitting & Drift
Two failure modes erode accuracy - one is baked in before launch, the other creeps in long after it:
psychologyOverfitting
The model memorized noise and past patterns that will not repeat - brilliant in training, poor live. Complex models chasing every historical wiggle fail when tomorrow behaves differently.
updateModel Drift
The market changed and the model did not - new competition, shifting mix, a post-renovation reality. Accuracy decays silently until someone checks the live error rate.
Track error where it hurts: measure accuracy by date and segment against the decisions it drives - a 5 percent error on the citywide-congress Saturday costs more than 15 percent on a random Tuesday.
translatePenjelasan dalam Bahasa Indonesiaexpand_more
Akurasi model dan validasi adalah disiplin yang mengubah prediksi dari opini bersimbol desimal menjadi ilmu terukur: forecast yang tidak diukur tidak bisa dipercaya. Enam metrik menilai model dari berbagai arah, backtesting menilai kejujurannya, dan dua mode kegagalan menggerogotinya diam-diam. Pelajaran ini membahas semuanya beserta praktik pelacakan galatnya.
- check_circleMAPE adalah standar industri: galat delapan persen berarti rata-rata meleset delapan persen; intuitif dibaca.
- check_circleMAE dan RMSE dihitung dalam kamar; RMSE menghukum miss besar lebih keras daripada miss kecil.
- check_circleForecast bias, yaitu over- atau under-forecast sistematis, adalah galat paling berbahaya karena menggandakan diri di tiap keputusan.
- check_circleAkurasi campuran menyembunyikan bencana segmen; skor total sehat bisa menutup forecast bisnis hari kerja yang keliru.
- check_circleJangan pernah menilai model pada data latihannya; tahan periode terbaru dan ramalkan seolah buta.
- check_circleOverfitting tertanam sebelum peluncuran; drift menyusup lama setelahnya saat pasar berubah tanpa model ikut berubah.
helpPertanyaan Umum (FAQ)
Metrik apa yang dipakai mengukur akurasi forecast hotel?
Enam metrik utama: MAPE atau rata-rata galat persentase yang intuitif dibaca, MAE dan RMSE yang dihitung dalam kamar dan menghukum miss besar, forecast bias untuk mendeteksi over- atau under-forecast sistematis, akurasi per segmen agar bencana tidak tertutup skor campuran, hit rate sebagai pandangan lolos-gagal, serta value score yang memberi bobot dampak pendapatan.
Apa itu backtesting dan walk-forward testing?
Backtesting menahan periode terbaru sebagai data uji, meramalkannya seolah model buta, lalu mencocokkannya dengan aktual secara jujur. Walk-forward mengulang latihan itu di banyak tanggal: latih, ramalkan, maju, ulangi, sehingga skor mencerminkan kondisi nyata seperti musim ramai dan sepi, event, data hilang, dan pickup terlambat. Satu bulan beruntung tidak membuktikan apa pun.
Apa perbedaan overfitting dan model drift?
Overfitting sudah tertanam sebelum peluncuran: model menghafal derau dan pola masa lalu yang tak akan berulang sehingga cemerlang saat latihan namun buruk saat live. Drift menyusup jauh setelah peluncuran: pasar berubah karena kompetitor baru atau bauran bergeser sementara model diam, dan akurasinya meluruh senyap hingga seseorang memeriksa galat live.
Mengapa akurasi perlu dilacak per tanggal dan segmen?
Karena galat tidak berharga sama: galat lima persen pada Sabtu congress citywide menghabiskan biaya lebih besar daripada galat lima belas persen pada Selasa acak. Ukur akurasi pada tanggal dan segmen yang keputusannya dipengaruhi model, agar perbaikan diarahkan ke tempat berdampak pendapatan terbesar.