Do you think it would be possible to have a 'full and frank' discussion about why search has not worked for the past 5 months and why no-one appears to be interested in getting this working again?

It is rumoured that it was removed in order to prevent Scoobynet members searching for threads where certain companies are shown to have acted in bad faith. Said companies may also be sponsors of Scoobynet banners (allegedly).

Can we please have some answers.

 

See my Fun thread, you can't hide much on the internet.

 

Nothing so juicy
The freeform nature of search was hammering the servers and since the volume of searches was so great, it was causing them to fall over.

Simon removed the feature to stop him being called every 20 minutes to tell him the servers had crashed again.

 

Neil,
Would it be possible to tweak the search to just look through the topic 'headings' as opposed to the freeform text?
Or has the search facility gone for good ?

Jamie

 

Jamie, I dunno m8

I do know that Simon had a whole team of scoobynetters looking at the problem for about a month. I'm sure a title search is possible, but without knowing the details of how it works at a system and application level I can't say if it's possible.

I think Evil Bevel helped out so he may be able to shed more light on things..

 

Neil, I'd be happy to share my 0.02 cents, but in the light of the first post, I realize my answers wouldn't make one bit of difference.

I think everyone wants to believe their own story, no ?

I must say I'm disgusted with the way the "question" was put forward.

Personally, I think it's an MI5/CIA cover up operation ...

Theo

 

Arr, the power of the Internet.

Ha hah!

 

I find it quite disheartening when a topic is approached like this......especially with angry face icons!!!!!!!!

This topic has been done to death by now, BUT......

The search was disabled due to the adverse effect it had on crashing the new scoobynet software......what would you rather have??

Due to time constraints and everyday life, no available time has been available for this to be remided.

It is a priority, once the above item can be resourced.

It has not been disabled purposefully, but for the only reason of KEEPING SCOOBYNET LIVE!!!!!!

I apologise on behalf of Scoobynet to Theo, for certain comments that have been made by individuals (this is the guy, who has spent hours working on the original search engine for this site...FOR FREE!!!!!!).

Regards,
Shaun.

[Edited by Shaun - 3/15/2002 3:29:45 AM]

 

He only asked for an "explanation" It may have been "done to death" but without the search facility how did he know ??

If this was posted at the top of the "bugs" section or in the post that stays at the top, there would probably be no more postings on this subject until it is (hopefully) resolved.

What I've seen in the last few months is a massive increase in new postings covering subjects that have already been discussed, it's a shame that people then don't get replied to and find the answers to their questions

[Edited by DAZ 4 - 3/15/2002 8:52:01 AM]

 

Here we go, again! Shoot the messenger. If anyone would like to pop over to 'Bugs' they'll see that there have been numerous requests for progress with regard to this problem and while there may have been statements in the past, it is difficult to determine the current position. (due to the lack of Search)

If search doesn't work under the new scoobynet software, then is it because of the sheer volume of posts (would purging the old threads help?) or is it because the latest software is not as stable as the previous one? (consider moving back?)

 

I think Shaun may have mistaken the angry face for the conspiracy theory rather than what I took it to be, an angry face for the fact that search does'nt work.

Scoobynet would crash in 10 seconds flat if we went back to the old software, so a move back is out of the question(search also killed that software if I recall correctly). Evil Bevel then wrote another search routine which then worked ok. However the old software was then hitting performance constraints in other areas so we had to move to the new software.

I'm not 100% sure what the in depth issues with it are, only that it's not a single day fix and would require a lot of recoding to get it working reliably. However, I do seem to remember something about an issue with the software Si is using, not anything to do with Simons coding. I suspect if I've remembered correctly that when the bug in the software is sorted out, then the search will be easy to fix. If the bug in the third parties software is'nt fixed then search will need to be rewritten.

Hope this clears things up, and if it does'nt then tough. Because that's all I can remember

[Edited by Neil Smalley - 3/15/2002 10:02:22 AM]

 

OK,
Since the discussion has started, can I ask a technical question?
Theo, I am assuming that the original search was an ISAPI component written in C++/ MFC and talking to MS SQL. Is that right? What was the problem that was causing instability?

 

MrDefence,

Please understand that I'm not really at liberty to elaborate on technical details regarding Scoobynet.

However, I can say this (since the old search is obsolete now).

The search engine that was written for the "old" BBS (read... the UBB software) never caused any problems what so ever(from day one in February 2001 till the last day). It was an ISAPI program written in Delphi5. UBB works with flat files, so it was a case of skimming/reading those files, nothing to do with MS SQL (I wish LOL)

It was fully multithreaded, and great care was taken not to consume too many clock cycles/disk access, so normal operations did never suffer from that search engine.

UBB then fell over because the sheer number of posts (nothing to do with the search engine), so a brand new software was written by Simon de Banke. Needless to say that the old search engine I wrote was useless from then on. Also needless to say that writing a BBS software from scratch is a MAJOR achievement, especially when (this has been measured) the average response time server-to-browser is 0.5 seconds (yup, half a second)...

Just wanted to clear this up before people think it was my little search thingy that caused the problems with UBB

Hope this answers your question

Theo

 

Sorry you emphasised the time "yup, half a second", was that stressing how slow it was ?

While not wishing to tell anyone how to do their jobs, especially for a free service, but free products such as MySQL is used on many websites (such as imdb.com) and return results from MASSIVE word indexed databases in circa 0.02 seconds (from not particularly large computer systems)

If it's not a too intrusive question, how many concurrent user sessions does Scoobynet have at peak times ?

PS. I offer any help required to get a MySQL solution running.

[Edited by DavidBrown - 3/15/2002 6:26:49 PM]

 

Theo
Thanks for your input. I guess that draws a veil over any discussion about the latest version on a public board.

David,
I don't doubt that you know how to make a MySQL product work wonders, but I don't think rehashing the whole product would be met with much enthusiasm, seing as it is only the search that isn't working.

Basically webbie, I am offering to help. If it is a case of resources the ball is in your court.

EDIT : Don't want to speak for anyone else - changed above sentence from "we are" to "I am".

[Edited by MrDeference - 3/15/2002 7:13:28 PM]

 

Now that is refreshing, proactive positive and helpful.

I just wish I knew what they were talking about.

Would it be possible to store the old postings on CD and sell it to the BBS members?

Signed

Computer illiterate Jerome

 

Actually, I'm told Scoobynet is already running MySQL, I assumed as much but wasn't sure. So no-rehashing is necessary.

 

>>I guess that draws a veil over any discussion about the latest version on a public board.

I'm sorry, I should have known better and trust you fully with all the tecnical details that I know of, especially since you have 12 posts and no email address...



Theo

[Edited by EvilBevel - 3/15/2002 11:21:21 PM]

 

David, Sorry I misunderstood. I inferred incorrectly.

Theo,
Love it... have you been wronged recently? I was stating a fact. You closed the discussion. Don't frown at me because you assume the worst of everyone.

Seems all has been said that needs to. Simon has my address and my (and David's) offer. I am not special, I was just offering to help the community rather than whinge, FFS.

 

Folks,

I don't want to stir the argument and forgive me if these points have previously been made/considered on the board.

I always found the old search facility to be very useful in terms of the wealth of technical information it was able to generate. Since joining scoobynet I have read some very useful and well written postings. Will these now be effectively lost among the vast number of threads ?
Also will we now see similar topic questions being regularly asked as a result of the fact that people do not have the facility to properly look back to find that their question has already been asked by others and answered umpteen times before in the past. The quality of the feedback to such questions may also suffer if people become sick of posting answers to questions which they have already given in the past.

I can accept the fact that this Search facility is no longer available and the reasoning behind it. However I do think that it would be useful to have some sort of separate archiving facility to view old posts either by separate database or creation and distribution of CDs. I know this would cost money but if a nominal subscription fee / charge were made then I'm sure there would be sufficient demand to generate more than enough income to compensate.

(well thats my tuppence worth anyway!)
Ian

 

Am I the only one who bothers to look back through recent topics around thirty days for info?

You can't be in THAT much of a hurry for info, surely?

But then again, patience is a virtue and all that.

I sometimes wonder how Simon has the patience himself to listen to people whinging on the board whilst he provides a free service for us to use, but then, that is my opinion only

Enough of my early morning ramblings, I must sleep, ready for another night shift

*wanders back off to the Muppet Forum*

 

Maybe a update to the info message when you click on search may be useful as it just say due to technical reasons. I may be way off the mark but couldnt there be a separate domain (scoobynet_archives.com ??) and the search button would divert you there where the archives are kept and limit the number of users accessing it at one time ie. login in like chat and restrict it to 5 users at a time. Okay it may slow the search time down and you may have to keep trying to access it but it may be a temporary solution.

Obviously I dont know how feasible this is and it would require work to set it up.

Just an idea

chrisp

 

"I sometimes wonder how Simon has the patience himself to listen to people whinging on the board whilst he provides a free service for us to use, but then, that is my opinion only "...

Because the members are here .The advertisers pay to advertise to us. We might use their services.. The more members the more exposure for them.. More advertising... more money for simon to live on and invest.... Its a 2 way thing. Search is a major priority. Everyone benefits in the long run.

Simon ,this is not a "Pop" at you just my opinion on the matter.


[Edited by Luke - 3/16/2002 6:51:31 PM]

[Edited by Luke - 3/16/2002 7:18:10 PM]

 

Pfff. What practical difference does it make? 0.5 seconds or 0.02 seconds, neither is very long. If you're woried about 0.48 seconds, your life is far too busy for you to hang around Scoobynet...

 

For someone so flippant, you're displaying a disproportionate level of ignorance.

Imagine you have 100 requests coming into your server every second (not an unusual volume) and one server process (for the sake of simplicity). If it takes you 0.5 seconds to serve each request your server is going to be on it's knees, as it can only serve 2 per second.

If you can improve your server such that it takes 0.01s per request then 100 requests CAN be maintained without grinding to a halt.

That's the practical difference.

I'm currently helping Theo with optimizing the MySQL search of this BBS.

[Edited by DavidBrown - 3/18/2002 3:49:10 PM]

 

Ah, but the statistic there was 'on average', not for an individual case. Therefore - to me - the way that is was phrased means that when, say, the average of 100 users are on-line, they only have to wait 0.5s each. Certainly I've only ever found occassionaly that it's anything other than 'click' & it's there.

Anyway, who cares, it's there & it works well enough. I'd like the search as well, but I'd prefer the site to be working without search than not at all. My only hope is the extra load 'cos the search isn't working (things be re-hashed more, etc.) doesn't cause the same problem eventually but fro a different angle.

As it seems to be a third-party problem, it's probably best to leave it to them. It would be a disaster if someone got it wrong & the whole thing stopped working

 

Same here. Since the BBS was moved from a whole table -> a single table per post within a thread, it appears quicker (due to rendering issues).

I guess it's like trying to get people through a door. One person will pass through easily, try to get two people through simultaneously and things will start to be more tricky.. and so forth until you have a log jam where no-one can get through due to the pressure.

I'm not dissing Scoobynet etc, I'm helping. But in perspective in the "instant" world of the internet (not to be confused with bandwidth issues) 0.5s is a long time to fulfil a request from the server.