Error in test_policy number of solutions

I got the following error running the test_policy in the local build for a hack solution:

But, i know i have only completed 2 hack challenges and that is what appears in the page where you can check your progress, as you can see in the following image:

So maybe is counting the total gherkin solutions rather than just the hack ones? (2 hack + 2 vbd + 1 new hack = 5) what should i do in this case?

Please update, rebase and try again, if it fails please send the commit url to see what happens

Are you deleting the branch when you see that fail?, because I want to see the branch to found a way to help you.

Hi @pastel-code he is running the test locally, I’m doing that too and I’m having the same problem.

Yes, i deleted the branch to keep testing, but i have already pushed again, still having the same error in the test policy locally like @perceived-doubt said.

Ok, I will use the commit of him to see how I can help, I’m not expert in that job, because is new, but I can try to test it and find a way to solve that

Ok reading the commit, searching in the folders and reading the policy I think that I found the reason why this happen, I will test I to check if I’m right and I will tell you.

1 Like

Ok, I test it, the solution is simple, the job load all files that you upload for that challenge, for that reason the job count the script as a valid solution, for that reason you must save the script in a subfolder named with your username, like the images used for the evidence.

@major-indication You have 5 sols in hack because in another challenge you upload script without the proper structure in folders, for that you must fix that creating a subfolder in your challenge and move the script to that folder.


you can see in that image that you put the script in the same folder that the feature


Please move the script to the subfolder how an evidence and the error must be fixed, how you have that error not only for the actual challenge you must fix the challenge that you solve before of that.

To put it in easy words hack = 5 = 3 feature + 2 script where the scripts are in the wrong place.

Hi @pastel-code I don’t think thats it, since I have only pushed my feature files and its corresponding png evidences plus my code solutions, and my unique solution are: 5 code, 5 hack, 5 VbD and the count of test_policy is

There is one VbD solution missing and one plus solution in hack.

Can you give me your username or send me the commit link to investigate that, your case may be different, because I test it with the structure of @major-indication.


And this is my result (I have 15 for hack, but sum 2 for the wrong structure) and I test it again with the proper structure:


And you can see that only count 1.

If you think that is not your case, you can make the commit, send me the link of the commit and I will help you too.

Hi @pastel-code I just pushed my branch, I also opened an issue a few hours ago. thank you for your help.

Hi, it seems that your problem is different and can be related with the moving of vulnhub from hack to vbd, it’s probable that you vbd solution for vulnhub count as hack solution, and this can be the error why you count of hack is 1 more than the expected.

Please open a new thread to to try solve that, because is a different problem that @major-indication presents.

I will try to check if that i say is true (because I only make a rapid looking and that can explain your case) because that is the only thing that I can found, because the job take the solution type with the path, and you can found 3 path [ ‘training/code’, ‘training/hack’, ‘training/vbd’ ].

And how the CI runs in the nix virtual system can be outdated with that change of vulnhub.

But the reality is that I must check that because the job is new and I’m trying to figure out why happens every error that you can report.

Got it, how should i send the fix to the previous solution? in a separated commit in my branch and do another MR? or how?

Make the fix and upload the fix following the commit syntax

And make the MR, later try again with your challenge

Everything is fixed now thank you very much @pastel-code :grinning:

@major-indication no problem, we are here to help :smile:

Hi, @pastel-code thank you, that makes sense so for now I just decide to push my VbD solutions before my hack solutions while this bug is taken care of. Thank you for your help.

Hi, @perceived-doubt I write the solution for you in the issue that was opened, you can check that and fix it, that was a problem related to the structure too (but with vulnhub).

1 Like