Dialog vs Eleven preview eval

Small sample size in preparation for comparing Dialog vs Multilingual V2

Overall

Multilingual V2 (Model B)

0.15

PlayDialog (Model A)

Answers

2. PlayDialog (Model A) is better

-1

-2. Multilingual V2 (Model B) is better

369 (41.00%)

84 (9.33%)

62 (6.89%)

87 (9.67%)

298 (33.11%)

Deep analysis

2. PlayDialog (Model A) is better

-1

-2. Multilingual V2 (Model B) is better

		Statistics
00:00 --:-- 180.wav 00:00 --:-- 180.wav	PlayDialog Multilingual V2	-0.50 1.40 0.62
00:00 --:-- 181.wav 00:00 --:-- 181.wav	PlayDialog Multilingual V2	-1.40 0.90 0.40
00:00 --:-- 182.wav 00:00 --:-- 182.wav	PlayDialog Multilingual V2	-0.50 1.27 0.56
00:00 --:-- 183.wav 00:00 --:-- 183.wav	PlayDialog Multilingual V2	-0.30 1.31 0.58
00:00 --:-- 184.wav 00:00 --:-- 184.wav	PlayDialog Multilingual V2	0.40 1.36 0.60
00:00 --:-- 185.wav 00:00 --:-- 185.wav	PlayDialog Multilingual V2	0.50 1.27 0.56
00:00 --:-- 186.wav 00:00 --:-- 186.wav	PlayDialog Multilingual V2	0.70 1.22 0.54
00:00 --:-- 187.wav 00:00 --:-- 187.wav	PlayDialog Multilingual V2	0.50 1.08 0.48

Answers

A1. Expressiveness

A2. Audio quality

A3. Pacing (e.g. rhythm, tempo, speed)

A4. Voice accuracy (e.g. pronunciation)

A5. Hallucinations

A6. Other...

414 (46.00%)

118 (13.11%)

297 (33.00%)

64 (7.11%)

1 (0.11%)

6 (0.67%)

Deep analysis

A1. Expressiveness

A2. Audio quality

A3. Pacing (e.g. rhythm, tempo, speed)

A4. Voice accuracy (e.g. pronunciation)

A5. Hallucinations

A6. Other...


00:00 --:-- 180.wav 00:00 --:-- 180.wav	PlayDialog Multilingual V2	Voice accuracy (e.g. pronunciation) Expressiveness Audio quality
00:00 --:-- 181.wav 00:00 --:-- 181.wav	PlayDialog Multilingual V2	Audio quality Voice accuracy (e.g. pronunciation) Pacing (e.g. rhythm, tempo, speed) 2 more...
00:00 --:-- 182.wav 00:00 --:-- 182.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Audio quality
00:00 --:-- 183.wav 00:00 --:-- 183.wav	PlayDialog Multilingual V2	Audio quality Expressiveness Voice accuracy (e.g. pronunciation) 2 more...
00:00 --:-- 184.wav 00:00 --:-- 184.wav	PlayDialog Multilingual V2	Expressiveness Pacing (e.g. rhythm, tempo, speed) Audio quality 2 more...
00:00 --:-- 185.wav 00:00 --:-- 185.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Audio quality 1 more...
00:00 --:-- 186.wav 00:00 --:-- 186.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Voice accuracy (e.g. pronunciation)
00:00 --:-- 187.wav 00:00 --:-- 187.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Voice accuracy (e.g. pronunciation) Expressiveness

Overall

Multilingual V2 (Model B)

0.15

PlayDialog (Model A)

Answers

2. PlayDialog (Model A) is better

-1

-2. Multilingual V2 (Model B) is better

369 (41.00%)

84 (9.33%)

62 (6.89%)

87 (9.67%)

298 (33.11%)

Deep analysis

2. PlayDialog (Model A) is better

-1

-2. Multilingual V2 (Model B) is better

		Statistics
00:00 --:-- 180.wav 00:00 --:-- 180.wav	PlayDialog Multilingual V2	-0.50 1.40 0.62
00:00 --:-- 181.wav 00:00 --:-- 181.wav	PlayDialog Multilingual V2	-1.40 0.90 0.40
00:00 --:-- 182.wav 00:00 --:-- 182.wav	PlayDialog Multilingual V2	-0.50 1.27 0.56
00:00 --:-- 183.wav 00:00 --:-- 183.wav	PlayDialog Multilingual V2	-0.30 1.31 0.58
00:00 --:-- 184.wav 00:00 --:-- 184.wav	PlayDialog Multilingual V2	0.40 1.36 0.60
00:00 --:-- 185.wav 00:00 --:-- 185.wav	PlayDialog Multilingual V2	0.50 1.27 0.56
00:00 --:-- 186.wav 00:00 --:-- 186.wav	PlayDialog Multilingual V2	0.70 1.22 0.54
00:00 --:-- 187.wav 00:00 --:-- 187.wav	PlayDialog Multilingual V2	0.50 1.08 0.48

Answers

A1. Expressiveness

A2. Audio quality

A3. Pacing (e.g. rhythm, tempo, speed)

A4. Voice accuracy (e.g. pronunciation)

A5. Hallucinations

A6. Other...

414 (46.00%)

118 (13.11%)

297 (33.00%)

64 (7.11%)

1 (0.11%)

6 (0.67%)

Deep analysis

A1. Expressiveness

A2. Audio quality

A3. Pacing (e.g. rhythm, tempo, speed)

A4. Voice accuracy (e.g. pronunciation)

A5. Hallucinations

A6. Other...


00:00 --:-- 180.wav 00:00 --:-- 180.wav	PlayDialog Multilingual V2	Voice accuracy (e.g. pronunciation) Expressiveness Audio quality
00:00 --:-- 181.wav 00:00 --:-- 181.wav	PlayDialog Multilingual V2	Audio quality Voice accuracy (e.g. pronunciation) Pacing (e.g. rhythm, tempo, speed) 2 more...
00:00 --:-- 182.wav 00:00 --:-- 182.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Audio quality
00:00 --:-- 183.wav 00:00 --:-- 183.wav	PlayDialog Multilingual V2	Audio quality Expressiveness Voice accuracy (e.g. pronunciation) 2 more...
00:00 --:-- 184.wav 00:00 --:-- 184.wav	PlayDialog Multilingual V2	Expressiveness Pacing (e.g. rhythm, tempo, speed) Audio quality 2 more...
00:00 --:-- 185.wav 00:00 --:-- 185.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Audio quality 1 more...
00:00 --:-- 186.wav 00:00 --:-- 186.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Expressiveness Voice accuracy (e.g. pronunciation)
00:00 --:-- 187.wav 00:00 --:-- 187.wav	PlayDialog Multilingual V2	Pacing (e.g. rhythm, tempo, speed) Voice accuracy (e.g. pronunciation) Expressiveness