IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
[WSH] - Display custom charts and statistics (incl. Last.fm), Experimental WSH Script
muzack
post Jul 6 2010, 12:49
Post #1





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



Custom Charts & Statistic (incl. last.fm charts)



About

Display you Last.fm statistics graphs in fb2k via WSH Panel Mod.
Get more advanced graphs and stats with a custom playcount database.



Notes

I recommend to test this on a clean new fb2k test installation (simple test theme is included)
The whole script grows over time of learning. I always tried to simplified and cleanup the code, but there are still some weird processes that need to optimize.
Maybe I overused window.Repaint() or didn't understand things right. If you know better solutions, please tell me.
The script runs stable on my win7 64bit maschine with tons of components. I don't know how it performs on 10.000+ playcounts ;-)
I don't give warranty on anything, you should know foobar2000 and a bit Javascript/SQL to understand what happen.
I wrote this for my own music library and to learn WSH scripting/SQLite. Maybe I store some redundant data that blowup the database, maybe my scripting style is a mess, it aims not to be a professional application it's a testcase.


General Basic Setup

Download the latest script package and extract it to your fb2k program folder.

Install foo_wsh_panel_mode, disable Safe Mode in Preferences and setup a view for the panel.
Load the file "scripts/custom_playback_charts_and_statistics_v*.txt" into the panel
Left mouse click on panelheadline/icon show you the main menu for the statistics.

Get Last.fm Charts

This is the easiest part wich only display your Last.fm charts
Set you last.fm username and API Key.
If you use marc2000 last.fm playcount sync script, or biography script, the key/username files are already there and charts should be displayed.
You can set a date range ( "one month" doesn't work for last.fm) and limit the items in charts.
In your %FB_PROFILE%/wsh_lastfm/ folder (foobar2000 profile path) you should find a lfm-chart.json file which hold an object with all chart data, parsed by the wsh script.
Thats's all. Hopefully smile.gif.
This works independently from the custom playcount statistic


Install local playcount database.

This is a bit complicated.
Ok - you can easily install the db from menu (see the first screenshot below) and it's ready to run. It places a playcount.sqlite db file in your fb2k profilefolder.
But your library should be accurate tagged to get the best results, so new files should be tagged right to keep track.
If you get files from different sources, it's natural that you need to retag them to fit's your need or you retag them later, whyever. If artist or title or album are changed you lose track of the statistic.
Note that the foobar2000 playbackcount statistic component work independent from this and playcounts differ from older files and may differ from newer files with custom playbackstatistic.
So to keep track as best as possible you need a unique id for artist, album, titles.
You can archieve that with an acoustic fingerprint of each single file and a masstagger script which turns your fingerprinted and well tagged artist and albumnames into unique ids via fb2k titleformat $crc32 function.
Find the masstagerscript inside the script folder.
BACKUP YOUR FILES AND/OR TEST A SMALL SET OF FILES!
See in scripts/custom_playback_statistic/install-custom-playcount-info.txt folder for some more information!

Extra
I've included a modification of the aero seekbar script, with an visual indicator when the track is counted.
Just import the aero-seekbar-mod-v0.2.txt in a WSH panel.




Required Components:

foo_uie_wsh_panel_mod
http://www.hydrogenaudio.org/forums/index....showtopic=70363

Recommended Components:

foo_playcount
http://www.foobar2000.org/components/view/foo_playcount

foo_masstag
http://www.foobar2000.org/components/view/foo_masstag

foo_biometrics
http://www.hydrogenaudio.org/forums/index....showtopic=65185

foo_new_file_stamper_mod
http://www.hydrogenaudio.org/forums/index....showtopic=68139

foo_audioscrobbler
http://www.foobar2000.org/components/view/foo_audioscrobbler

foo_httpcontrol (experimental)
http://www.hydrogenaudio.org/forums/index....showtopic=62218

SQLite Manager - Firefox Extension
https://addons.mozilla.org/en-US/firefox/addon/5817/

ToDO

lots of things, cleanup and simplify code
some display glitches

Credits

Thanks to the WSH Panel developer and the people who share their scripts.
Some parts based of the script collection from marc2000 Last.fm Playback Statistic import
Some helper function where found on the internet. A few are written by myself.

As I said before this is a study and experimental. I don't know how it behave under a different environment than mine, I try to simplify the install process.
I don't want to force you to do all this tagging stuff. It's just my way.

Good Luck!

PS: sorry for my english ;-)
Go to the top of the page
+Quote Post
kabuki
post Jul 6 2010, 14:20
Post #2





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



this will please to a lot of people...
thank you for that. I was secretly hopping something like that.
i will test it asap.
cheers

This post has been edited by kabuki: Jul 6 2010, 14:22
Go to the top of the page
+Quote Post
marc2003
post Jul 6 2010, 23:32
Post #3





Group: Members
Posts: 5322
Joined: 27-January 05
Member No.: 19379



good job. i've only tested the last.fm side of things but it works well enough. i did think about doing this myself but was too lazy to get around to it. tongue.gif

EDIT: actually i do see a few problems. the bar charts aren't scaled properly all the time. but the behaviour is pretty random.

This post has been edited by marc2003: Jul 7 2010, 00:19
Go to the top of the page
+Quote Post
muzack
post Jul 7 2010, 07:24
Post #4





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (marc2003 @ Jul 7 2010, 00:32) *
EDIT: actually i do see a few problems. the bar charts aren't scaled properly all the time. but the behaviour is pretty random.


the only thing i noticed at lfm side is, the labels (artist/artist ~ track/artist ~ album) jumping a bit (alignment) while switching through the 3 states.
maybe because the previous chart will parsed again on switching, while the new one is loaded.
Go to the top of the page
+Quote Post
kabuki
post Jul 7 2010, 14:37
Post #5





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



QUOTE (marc2003 @ Jul 7 2010, 00:32) *
EDIT: actually i do see a few problems. the bar charts aren't scaled properly all the time. but the behaviour is pretty random.



is this normal scale? strange behavior: same scale from 163 to 365 or from 1 to 4.

This post has been edited by kabuki: Jul 7 2010, 14:42
Go to the top of the page
+Quote Post
muzack
post Jul 7 2010, 15:23
Post #6





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (kabuki @ Jul 7 2010, 15:37) *
is this normal scale? strange behavior: same scale from 163 to 365 or from 1 to 4.


yes this is normal - last.fm behave the same way:


this happens when you got big gaps between the counts.
it has to do with the calculation of bar width

Ok, i've revisited the code right now and there is a line
CODE
var min_width = gr.CalcTextWidth(min_widthValue, g_font)*11;
(in function paint_charts() )
if you remove the *11 it looks nicer.
I'm sure there was a reason, to calculate it this way, but I can't remember smile.gif

Edit: if you display a chart like above in a narrow panel it can happen, that the placount numbers disappear (depending on width of the string).
the multiplication ensures that there is enough space even if there are big jumps between PC
that was the reason smile.gif

This post has been edited by muzack: Jul 7 2010, 15:37
Go to the top of the page
+Quote Post
kabuki
post Jul 7 2010, 15:50
Post #7





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



QUOTE (muzack @ Jul 7 2010, 16:23) *
Edit: if you display a chart like above in a narrow panel it can happen, that the placount numbers disappear (depending on width of the string).
the multiplication ensures that there is enough space even if there are big jumps between PC
that was the reason smile.gif
yep, i just notice that.
Is one of the reason (apart that i am trying to reproduce LastFM scheme) why i replace alignement Right (far) by Left (Near).
Anyway, thank you for pointing me that part of script.
Go to the top of the page
+Quote Post
kabuki
post Jul 7 2010, 17:28
Post #8





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



A question (classical one, i guess) about using local playcount.
as i understand your notice, the local playcount mode uses playcount.sqlite. A new file generated when "installing custom playcount statistic" and poulated after 50% of playing time.

all my files are already tagged with LAST_PLAYED FIRST_PLAYED PLAY_COUNTER etc.
is there a way to use those tags, instead of another database?
maybe this question is not relevant to your stuff? Sorry, i don't want to be boring.
Go to the top of the page
+Quote Post
muzack
post Jul 7 2010, 17:53
Post #9





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (kabuki @ Jul 7 2010, 18:28) *
all my files are already tagged with LAST_PLAYED FIRST_PLAYED PLAY_COUNTER etc.
is there a way to use those tags, instead of another database?

foobar2000 playback stat just add +1 to the track playcount and it tracks just first- and last played date.
in between there is nothing - you have no timeline when a track was played, that's the reason why I did this extra database.
it was just kind of research for myself but since a month it runs stable so it's ok to start a third playing statistic smile.gif

BTW. Today I found out that on last.fm playground they have a similar/much better graph for your listening timeline smile.gif
http://playground.last.fm/demo/timeline
Go to the top of the page
+Quote Post
kabuki
post Jul 7 2010, 21:49
Post #10





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



Maybe i am wrong,
Haven't you forgot to define "var g_textcolor" & "var g_textcolor_hl" & "var g_font"?
Or maybe those variables are defined in another file.


EDIT: found definition in common.js // color depends on DUI color
wink.gif

This post has been edited by kabuki: Jul 7 2010, 22:42
Go to the top of the page
+Quote Post
kabuki
post Jul 8 2010, 17:48
Post #11





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



Hi Muzack,

What was your thought having "limits chart" (set to 5,10,15 or 20) ?
Do you display sometimes only 5 charts? (in that way, bars's row is very high, because [bars row = window height / limit chart] ).

I can already change part of your code to make bar row fixed, this is not complicated at all. Aswell as changing limits to 10,30,50 or 100 if i want that. (thank you for an easy code wink.gif )

So now you can imagine my next "problem"
If bar row are set to 15px and limit of charts set to 100 or 200, wsh panel will only display X first bars charts depending on my window height; logical wink.gif
Will it be hard to have the possibility to scroll charts?

In other words, i don't ask you to make that change because of my taste, but to point me in the good part of the code to change, if i want to make it by my own.
Or maybe this suggestion could have interest for more than me, and you could maybe plan to include that change.

Cheers
Go to the top of the page
+Quote Post
muzack
post Jul 8 2010, 19:40
Post #12





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



I thought about scrolling, but it was a bit too complex at this point, and I was fine with the limit.
There are some examples outside how to scroll inside the wsh panel.
marc2000 last.fm biography viewer use that, so you may look at his script examples.
but it's lot of juggling with variables ^^

PS: because of initially scaling the bars I think it need some/lot code rewrite.

This post has been edited by muzack: Jul 8 2010, 19:42
Go to the top of the page
+Quote Post
marc2003
post Jul 8 2010, 20:58
Post #13





Group: Members
Posts: 5322
Joined: 27-January 05
Member No.: 19379



the simplest way to scroll is using the mouse wheel. something like this...

CODE
var offset = 0; //this variable should be added to the y co-ordinate of every text/image/draw element in on_paint

function on_size() {
  ww = window.width;
  wh = window.height;
  offset = 0; //need to reset position when panel is resized
}

function on_paint(gr) {
  gr.drawWhatever(x, y + offset,...........
}

function on_mouse_wheel(delta) {
  offset += -delta;
}


now that only scrolls one pixel at a time so you'd probably want to use a multiple with the delta.

CODE
offset += -delta * 15; //15 pixels


at the moment that code would scroll in either direction without stopping. now you have other things to consider in the on_mouse_wheel function.

-you don't want to scroll when the panel height is already bigger than your content.
-you don't want to scroll upwards when at the top
-you need to know the height of your content so you can stop scrolling at the bottom

hopefully that should help kabuki get started. smile.gif

This post has been edited by marc2003: Jul 8 2010, 21:03
Go to the top of the page
+Quote Post
kabuki
post Jul 8 2010, 21:05
Post #14





Group: Members
Posts: 242
Joined: 22-June 06
Member No.: 32129



thank muzak for answer and direction where to look at.
thank marc2003 for examples.
I'll try. At first sight, JScript was like chinese for me (i am european), now it seams more like italian (which i don't speak neither), but slowly, and step by step, i reach to do some change. So with your answer, marc, i will go further.
Have a nice week-end.
Go to the top of the page
+Quote Post
novembre
post Jul 9 2010, 23:27
Post #15





Group: Members
Posts: 132
Joined: 6-December 08
From: Italy
Member No.: 63998



Interesting!
Most of my files are tagged with PLAY_STAMP, is there a way to import this info in the custom db?
Go to the top of the page
+Quote Post
marc2003
post Jul 10 2010, 00:26
Post #16





Group: Members
Posts: 5322
Joined: 27-January 05
Member No.: 19379



QUOTE (muzack @ Jul 7 2010, 15:23) *
yes this is normal - last.fm behave the same way:


just because they're doing it wrong doesn't mean it's normal. i don't even know how to make it that wrong. doing it right is considerably easier. tongue.gif





Go to the top of the page
+Quote Post
muzack
post Jul 10 2010, 07:51
Post #17





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (marc2003 @ Jul 10 2010, 01:26) *
QUOTE (muzack @ Jul 7 2010, 15:23) *
yes this is normal - last.fm behave the same way:

just because they're doing it wrong doesn't mean it's normal. i don't even know how to make it that wrong. doing it right is considerably easier. tongue.gif


@marc2000 yes and now, i don't think they do it wrong (but maybe I did ^^) because reflecting a jump from 10000 plays to 100,10, 1 plays on 200px width must result in same bar width on lower results (if there is a threshold of display minium width with numbers inside)
anyway - you lfm charts are much easier than mine weird calculations and behave more smoother. Good job (again)!

@novembre sorry no way to import. because the play_stamp reflect just one date (first/last listen/added/ date - and there is no timeline)
My custom playcount is a local "alternative" to last.fm and track each play with all need (and more) parameters to display a graph...
Go to the top of the page
+Quote Post
novembre
post Jul 10 2010, 11:18
Post #18





Group: Members
Posts: 132
Joined: 6-December 08
From: Italy
Member No.: 63998



QUOTE (muzack @ Jul 10 2010, 07:51) *
@novembre sorry no way to import. because the play_stamp reflect just one date (first/last listen/added/ date - and there is no timeline)
My custom playcount is a local "alternative" to last.fm and track each play with all need (and more) parameters to display a graph...

Sorry, I should have been more precise in the previous post. PLAY_STAMP tag is created by foo_playback_custom and it has the date/time of every play like this:
CODE
2009-01-21 19:22:47; 2009-10-10 14:41:51; 2009-12-01 22:59:14; 2010-04-10 14:08:25; 2010-05-06 13:19:50

What about in this case?
Go to the top of the page
+Quote Post
muzack
post Jul 10 2010, 15:14
Post #19





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (novembre @ Jul 10 2010, 12:18) *
QUOTE (muzack @ Jul 10 2010, 07:51) *
@novembre sorry no way to import. because the play_stamp reflect just one date (first/last listen/added/ date - and there is no timeline)
My custom playcount is a local "alternative" to last.fm and track each play with all need (and more) parameters to display a graph...

Sorry, I should have been more precise in the previous post. PLAY_STAMP tag is created by foo_playback_custom and it has the date/time of every play like this:
CODE
2009-01-21 19:22:47; 2009-10-10 14:41:51; 2009-12-01 22:59:14; 2010-04-10 14:08:25; 2010-05-06 13:19:50

What about in this case?


oh that's nice - didn't know about that ^^
well then it is possible over text tools of foobar, but it's a bit quirky.
You can export your whole lib into a textfile
so you got a dataset of title info, + comma seperated PLAY_STAMP

then this set must processed a second time by a scriptlanguage (php, VB...) to split PLAY_STAMP values and assign those to the trackinformation and write a sql to import into (see database table layout)
so
1. let foobar export your lib
2. transform the data with any scripting language to sqlite importformat
3. import into playcount.sqlite

for the first part I could give you the code later. need to go now.

Go to the top of the page
+Quote Post
muzack
post Jul 11 2010, 12:49
Post #20





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



To Import PLAY_STAMP timeline
export your lib via http://www.foobar2000.org/components/view/foo_texttools in a textfile
enter the following syntax into texttools and save it as file

CODE
$if(%FINGERPRINT_FOOID%,$crc32(%FINGERPRINT_FOOID%),$crc32($lower([%artist%][%album%][%title%])))|[%artist%]|[%album%]|[%title%]|$if(%FOO_ARTIST_ID%,%FOO_ARTIST_ID%,$crc32($lower([%artist%])))|$if(%FOO_ALBUM_ID%,%FOO_ALBUM_ID%,$crc32($lower([%album%])))|
%play_count%|%first_played%|%last_played%|[%PLAY_STAMP%]

(Scheme: Trackid|artist|album|title|artistid|albumid|playcount|first played|last played|play stamp )

now process the textfile with php or any other language
pseudo php code:
CODE
$csv = "1234|artist|album...";
// transform csv to array
/*
$csvArr[0]['trackid'] = "1234";
...
$csvArr[0]['play_stamp'] = "date1,date2,date3";
$csvArr[1]['trackid'] = "3456";
...
$csvArr[1]['play_stamp'] = "date1,date2,date3";
*/

// loop over array
foreach($csvArr as $key => $value)
{
  $playstampArr = explode(",",$csvArr[$key][play_stamp]);
  foreach($playstampArr as $k => $v){
// build correct SQL statement with (name/value pairs)
   $sql[] = "INSERT OR REPLACE INTO playcount(trackid,artist,album...playtime) VALUES (".$csvArr[$key][trackid].",'".$csvArr[$key][artist]."','".$csvArr[$key][album]."','".$v."')"; // $v is the playdate value
}
}

echo implode("\n",$sql);


now import the text into sqlite db with firefox extension sqlite manager
you have to process the textvalues for the sqlstatement and mask ' in titlenames with double ' something like this
It's over => It''s over

That's the way I would do it.
cheers
Go to the top of the page
+Quote Post
novembre
post Jul 12 2010, 15:55
Post #21





Group: Members
Posts: 132
Joined: 6-December 08
From: Italy
Member No.: 63998



Awesome! w00t.gif


Got it working with no SQL or PHP knowledge, I used foo_text_tools, a text editor, a spreadsheet and Firefox SQLite Manager extension. Thanks for the hints, muzack!
Go to the top of the page
+Quote Post
muzack
post Jul 12 2010, 20:24
Post #22





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (novembre @ Jul 12 2010, 16:55) *
Awesome! w00t.gif
Got it working with no SQL or PHP knowledge, I used foo_text_tools, a text editor, a spreadsheet and Firefox SQLite Manager extension. Thanks for the hints, muzack!


wow, nice graph smile.gif - how does it perform to display the whole timerange in foobar? is it fast enough or does it hang a bit?

Go to the top of the page
+Quote Post
novembre
post Jul 12 2010, 21:24
Post #23





Group: Members
Posts: 132
Joined: 6-December 08
From: Italy
Member No.: 63998



It seems fast (less than 1 sec even with artists with over 1K plays) but I've only tested it on a quad-core, tomorrow I'll try on a slower dual-core.
Two things I've noticed, maybe due to an imperfect import:
1) sometimes the graph doesn't reach the full height indicated by the background gray lines, i.e. the max play count per day is 30 and the graph stays in the bottom half of the window because the gray line it's at 97 plays
2) with date range set to overall, the graph always starts at the first played date ever in the database (in my case 26/02/2008) , maybe you could add the possibility to switch from this "global" overall to a "relative" overall, in which the graphs starts from the first played (or added) date of the artist/track/album

This post has been edited by novembre: Jul 12 2010, 21:25
Go to the top of the page
+Quote Post
muzack
post Jul 12 2010, 22:07
Post #24





Group: Members
Posts: 113
Joined: 14-December 08
Member No.: 64497



QUOTE (novembre @ Jul 12 2010, 22:24) *
It seems fast (less than 1 sec even with artists with over 1K plays) but I've only tested it on a quad-core, tomorrow I'll try on a slower dual-core.
Two things I've noticed, maybe due to an imperfect import:
1) sometimes the graph doesn't reach the full height indicated by the background gray lines, i.e. the max play count per day is 30 and the graph stays in the bottom half of the window because the gray line it's at 97 plays
2) with date range set to overall, the graph always starts at the first played date ever in the database (in my case 26/02/2008) , maybe you could add the possibility to switch from this "global" overall to a "relative" overall, in which the graphs starts from the first played (or added) date of the artist/track/album

thanks for reply
if you like send me your database (via pm and mediafire or something like that) - I like to test it with a bigger db.
I think the sql process is fast enough, just the parsing of the csv file with Jscript is the bottleneck (on slower computers) but I don't know.
to 1) it could depend on the import because the data is nonlinear in database and maybe it needs an extra sort. But I could test it with your db.
to 2) interesting point - I think about it.
I thinked about a way of calendar like browsing, but that's an universe for it's own.
Go to the top of the page
+Quote Post
Kurtegikov
post Jul 13 2010, 10:43
Post #25





Group: Members
Posts: 8
Joined: 13-July 10
Member No.: 82231



Can you please share that buttons (WSH?) panel (code and pics) with Discogs, SongMeanings and Musicbraiz buttons that can be seen in first screenshot? Looks very nice, would like to try it.
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th April 2015 - 17:05