pushed to master at Jaeha.Choi/WikiPageview_2.0
- 3d5d0209a4 Fix missing values when processing
Changed the goroutine behavior. Files now use one thread-safe map to store all daily pageviews, then insert data to the database, to improve the execution time and to prevent deadlocks on the database side. The last writeHour will execute the writeMonthly. Due to new changes, only the executions that start and end at midnight will work without overwriting any data. This is because the current approach is initializing the base array to 0, causing the insert to overwrite the existing data. Also, the duplicates must be checked. Current observation suggests that the duplicates are caused by a case-insensitive table. Need to investigate my.ini settings on Windows to fix this issue.
Signed-off-by: Jaeha Choi <firstname.lastname@example.org>
2 months ago