Language rewrite - any success stories?

On Jan 28, 10:06 pm, awhite <spud... DeleteThis @iinet.net.au> wrote:
> A ramble.
>
> In finding a new reason not to work hard at my game, I've lately become
> enamoured of Common Lisp.
>
> Don't get me wrong... C++ is a great language. It's a fun language. It's
> very very powerful, and I've invested a lot of money in books on
> techniques, structure, coding styles, "don't do that" practices, and
> others. But it's quite slow to develop in. An edit-compile-link-test run
> takes a lot of time. Tools can speed that up, but on linux (my primary
> platform), there's really only g++ and icc.
>
> Common Lisp is fun, quick, and just as capable for roguelike development
> (a couple of quite good roguelikes have been written in it).
>
> Conventional wisdom is to never rewrite, but to refactor, refactor,
> refactor. I agree with that, so my current codebase is uglier than
> necessary - it still contains vestiges of being written for multiple
> executable targets (server, client, and client/server), multiple players,
> and even multiple simultaneous displays (SDL and curses, though I'm
> tempted to keep those).
>
> But the edit/compile/link/test cycle is getting me down. I've been
> looking for a faster-to-develop language, and Paul Graham's essays have
> convinced me that I'm missing out on something big.
>
> So... Has anyone else tried a total rewrite in a different language? And
> succeeded? There are probably a fair few attempts out there for various
> projects - crawl5 (short-lived as it was) and langband come to mind. Has
> anyone else succeeded?
>
> Adam

I'm gonna be a jerk and enamour on ways you can improve your
compilation speed rather than language changes. The first is
precompiled headers. If you have g++ compile a header file with the
option "-x c++-header", the resultant file will have the
extension .gch and can be used to significantly improve build speed.
You just #include naturally and then the header will be used instead.
However, you can only use ONE precompiled header per compile, so you
must choose the header wisely (project-wide stuff like global
definitions and stuff is a good choice). The inclusion also has to be
the first non-preprocessor thing to occur in the file.

The second is to use separate static libraries. It will speed up your
project link time significantly if the entire project doesn't need to
be relinked entirely.

Personally, though, if you're changing languages, it's probably a lot
like making a new RL, unless they are very similar (e.g. C -> C++, or C
++ -> Java on a good day). C++ to Common Lisp is definitely not an
easy change in any way.

I've seen programs go through language changes before, though, and
often for the better. But it's a painstaking process, and it can be a
very long time before you get back to where you were before.
 >> Stay informed about: Language rewrite - any success stories? 

On 29 Jan, 07:22, Krice <pau... RemoveThis @mbnet.fi> wrote:
> On 29 tammi, 07:06, awhite <spud... RemoveThis @iinet.net.au> wrote:
>
> > Tools can speed that up, but on linux (my primary
> > platform), there's really only g++ and icc.
>
> Get Windows and Visual C++. It's incredibly fast. VC
> can rebuild Kaduria in less than minute!

I think this says more about the amount of code in Kaduria than the
speed of VC++.

-the ru
 >> Stay informed about: Language rewrite - any success stories? 

On 29 tammi, 09:26, "Ulf Åström" <ulf.ast....TakeThisOut@gmail.com> wrote:
> I think this says more about the amount of code in Kaduria than the
> speed of VC++.

Before VC++ I was using DevC++ (with GCC) and it took much longer
to rebuild, I think it was about 7 minutes. Besides DevC++ just
could't keep track of changed files so you had to rebuild all the
time which was very annoying. VC++ knows what modules need to be
updated when you change something. It's fast, trust me:)
 >> Stay informed about: Language rewrite - any success stories? 

On 29 Jan, 07:43, coppro <ride....DeleteThis@gmail.com> wrote:
> On Jan 28, 10:06 pm, awhite <spud....DeleteThis@iinet.net.au> wrote:
>
> > So... Has anyone else tried a total rewrite in a different language? And
> > succeeded? There are probably a fair few attempts out there for various
> > projects - crawl5 (short-lived as it was) and langband come to mind. Has
> > anyone else succeeded?
>
> I've seen programs go through language changes before, though, and
> often for the better. But it's a painstaking process, and it can be a
> very long time before you get back to where you were before.

Simply rewriting the code is a very straight-forward process. The real
problem is keeping motivation up when you are halfway through. The
rewrite isn't complete enough to be playable, but the old code is too
ugly to resume working on, so instead you end up with *no* code being
written.

Also, there is no such thing as perfect code (at least not once we
move beyond trivial 10-line school assignments) nor perfect program
design. Even if you rewrite your game to fix problems in the old code,
you *will* run into new ones. Starting over just to "get it right" can
be fatal; you go from a working game with poor code to no game at all.

The only thing that helps is discipline. Keep your head cool and plan
ahead *before* you start implementing that ultra-cool feature. Go
through your code once in a while to document new functions and remove
things that are no longer valid.

I haven't had much experience with LISP, but I'm really curious what
makes you believe developing in it would be faster than in C++.
Speeding up the compile process of your current code (e.g. by using
makefiles so you only recompile *changed* classes, etc) seems like the
obvious solution.

-the ru
 >> Stay informed about: Language rewrite - any success stories? 

On 29 Jan, 10:11, Adam White <spud... DeleteThis @iinet.net.au> wrote:
> Lisp is new and (because of that) more fun, so I'm more motivated.

I'd hardly call it "new".

http://en.wikipedia.org/wiki/Lisp_%28programming_language%29

.... or perhaps you mean "new" *to you*; in that case, disregard this
message. ;-]

-the ru
 >> Stay informed about: Language rewrite - any success stories? 

On 29 Jan, 09:53, Kornel Kisielewicz <admin RemoveThis @nospam_chaosforge.org>
wrote:
> In a state of madness Krice wrote the following :
> > On 29 tammi, 07:06, awhite <spud... RemoveThis @iinet.net.au> wrote:
> >> Tools can speed that up, but on linux (my primary
> >> platform), there's really only g++ and icc.
>
> > Get Windows and Visual C++. It's incredibly fast. VC
> > can rebuild Kaduria in less than minute!
>
> Well, a complete DoomRL rebuilt including Valkyrie and the wad creation
> program, lua precompilation and all is... 4 seconds . That's another
> reason why I love FreePascal ^_^.

TSL (31k lines pure C) right now compiles in 24 seconds for me, with
GCC/Linux on a 400 MHz machine.

This could probably be sped up a lot as well by selecting what parts
to compile; now I recompile everything every time. The delay is still
small enough to not bother me, though.

-the ru
 >> Stay informed about: Language rewrite - any success stories? 

On 29 tammi, 10:55, g... DeleteThis @mail.ru (Timofei Shatrov) wrote:
> Without any pre-compiled object files? I don't believe it. Well,
> unless Kaduria codebase is really small.

Something like 150 source files. I could test how long it
actually takes and what is the current SLOC.
I don't use precompiled files.
 >> Stay informed about: Language rewrite - any success stories? 

On Mon, 28 Jan 2008 22:43:00 -0800, coppro wrote:

[on c++ compilation speed]
>
> I'm gonna be a jerk and enamour

<pedant> elaborate? </pedant>

> on ways you can improve your compilation
> speed rather than language changes. The first is precompiled headers. If
> you have g++ compile a header file with the option "-x c++-header", the
> resultant file will have the extension .gch and can be used to
> significantly improve build speed. You just #include naturally and then
> the header will be used instead. However, you can only use ONE
> precompiled header per compile, so you must choose the header wisely
> (project-wide stuff like global definitions and stuff is a good choice).
> The inclusion also has to be the first non-preprocessor thing to occur
> in the file.

Yes, I realise that there are tools to speed up compilation. For example,
I used to find (using pure C on windows) that bcc was a bajillion times
faster than djgpp. G++ isn't the fastest compiler out there. VC8 seems
quicker, on windows.

But I'm still working around the problem that C++ (and reputedly java)
uses a lot of boiler plate code. I'm more looking for a solution to make
the language do the grunt work for me (lisp macros).

That's the reason I moved from C to C++ - I got a fair way into making C
into an object oriented language (via tables of pointers to functions,
casting pointers to structs to void * and back), before I realised that
my time would be better spent using an object-oriented language from the
get-go.


> Personally, though, if you're changing languages, it's probably a lot
> like making a new RL, unless they are very similar (e.g. C -> C++, or C
> ++ -> Java on a good day). C++ to Common Lisp is definitely not an easy
> change in any way.

Yes, and I'm scared of "second system effect" - making the second system
so generic and extensible that it never actually achieves anything.

>
> I've seen programs go through language changes before, though, and often
> for the better. But it's a painstaking process, and it can be a very
> long time before you get back to where you were before.

But I'm nowhere at the moment - I haven't released anything
Unlike Krice, I realise that until I actually release a playable *game*
any talk is just handwaving.


A
 >> Stay informed about: Language rewrite - any success stories? 

On Mon, 28 Jan 2008 23:26:22 -0800, Ulf Åström wrote:

> On 29 Jan, 07:22, Krice <pau....RemoveThis@mbnet.fi> wrote:
>> On 29 tammi, 07:06, awhite <spud....RemoveThis@iinet.net.au> wrote:
>>
>> > Tools can speed that up, but on linux (my primary platform), there's
>> > really only g++ and icc.
>>
>> Get Windows and Visual C++. It's incredibly fast. VC can rebuild
>> Kaduria in less than minute!
>
> I think this says more about the amount of code in Kaduria than the
> speed of VC++.
>

Damnit! Beat me to the punch!

A
 >> Stay informed about: Language rewrite - any success stories? 

On Mon, 28 Jan 2008 23:51:08 -0800, Ulf Åström wrote:

[on rewriting]

> Simply rewriting the code is a very straight-forward process. The real
> problem is keeping motivation up when you are halfway through. The
> rewrite isn't complete enough to be playable, but the old code is too
> ugly to resume working on, so instead you end up with *no* code being
> written.
>

Yes - very very valid. But it's just as valid in a major refactor.


> The only thing that helps is discipline. Keep your head cool and plan
> ahead *before* you start implementing that ultra-cool feature. Go
> through your code once in a while to document new functions and remove
> things that are no longer valid.

Oh, that's not as much as a problem, as the time it takes to implement.

> I haven't had much experience with LISP, but I'm really curious what
> makes you believe developing in it would be faster than in C++.

stupid example below. I'm very much a lisp newbie, so it's probably very
flawed.

Using the REPL (Read Evaluate Print Loop) just seems faster, as
essentially there's no real difference between writing a program, and
using the debugger. You can write a function and debug it straight away.
No need to stop, compile, fire up the debugger, edit, recompile, fire up
the debugger, ad nauseum. That's the bit about c++ that's currently
depressing me.

In Lisp, you are literally in the debugger the whole time.

> Speeding
> up the compile process of your current code (e.g. by using makefiles so
> you only recompile *changed* classes, etc) seems like the obvious
> solution.

It's *a* solution. But I'm starting to think that C++ itself requires a
lot of boilerplate for its flexibility, and separate compilation model.
If C++ didn't support separate compilation, and (for historical reasons)
support it via name mangling (to use the standard linker), it could have
a true module system, and be a hell of a lot faster. ( <-- proof by
assertion, I know)

Take a very stupid example, from another project I've been thinking of:

;; (defstruct prototype ...) snipped
;; (defvar *monsters* (make-hash-table)) snipped

(defstruct mini
proto ; prototype definition
pos ; position
health ; current health
effects) ; effects in place

(defun create-mini (m)
"Create a mini from a prototype"
(let (p (gethash m *monsters*))
(make-mini p '(nil nil) (prototype-maxhealth p) nil)))


Essentially, that retrieves the "mini" prototype from a central database
(prototype pattern, for the GoF), and sets health to be equal to the mini
standard.

The "defstruct mini" macro creates member accessors, a constructor, truth
test, and lots more. It's concise.

How much boilerplate would that take to implement in C++ ?
The accessors, constructor, "rule of 3", plus memory management
(via boost::smart_ptr<> is my preferred option) would swell that out
immensely!

I think C# is a step in the right direction for the language - properties
and delegates are tres cool.


Adam
 >> Stay informed about: Language rewrite - any success stories? 

On Jan 28, 11:06 pm, awhite <spud....TakeThisOut@iinet.net.au> wrote:
> So... Has anyone else tried a total rewrite in a different language? And
> succeeded? There are probably a fair few attempts out there for various
> projects - crawl5 (short-lived as it was) and langband come to mind. Has
> anyone else succeeded?

Depends on what you mean by "rewrite" and "succeed." How big does a
program have to be before recoding it is considered a "rewrite"? I've
done 13 rewrites of my game in 5 different languages, but very few of
them could be considered major. I'd say I had one good version of my
game in QBasic, and two good versions in C++.

As for success, what determines that? Is success just being able to
have the same functionality with a different language, despite how
limited that functionality was? Because if that's the case, then yes,
I've been successful. It was only recently that I surpassed the
features of my QBasic version (sad, I know, but I've had to keep a
large scope in mind when adding things to the C++ version).

Even so, I still consider my rewrite a successful one. Despite having
VERY grandiose plans (I'm already planning three sequels to my current
project), I've set clear goals and am steadily accomplishing them.
I'm planning on releasing the first demo (ten levels to explore, a few
monsters and an end boss) within a week, so you all will be able to
judge how "successful" I've been. ^_^

--
Gamer_2k4
 >> Stay informed about: Language rewrite - any success stories? 

On Mon, 28 Jan 2008 22:22:06 -0800 (PST), Krice <paulkp DeleteThis @mbnet.fi> tried to
confuse everyone with this message:

>On 29 tammi, 07:06, awhite <spud... DeleteThis @iinet.net.au> wrote:
>> Tools can speed that up, but on linux (my primary
>> platform), there's really only g++ and icc.
>
>Get Windows and Visual C++. It's incredibly fast. VC
>can rebuild Kaduria in less than minute!

Without any pre-compiled object files? I don't believe it. Well,
unless Kaduria codebase is really small.

--
|Don't believe this - you're not worthless ,gr---------.ru
|It's us against millions and we can't take them all... | ue il |
|But we can take them on! | @ma |
| (A Wilhelm Scream - The Rip) |______________|
 >> Stay informed about: Language rewrite - any success stories? 

At 29 Jan 2008 05:06:22 GMT,
awhite wrote:

> But the edit/compile/link/test cycle is getting me down. I've been
> looking for a faster-to-develop language, and Paul Graham's essays have
> convinced me that I'm missing out on something big.

Write a 7drl in it. See how it works out, then you can start to extend
it with the ideas you already have in your current project (or you have
planned for it). This way you avoid two problems: the lack of motivation
while "only rewriting", and the illussion that rewriting is going to take
less time/effort than writing from scratch.

Another alternative to rewriting that I've seen working is to interface
your existing code with the language you want to move to, and then slowly
move parts of code to the other side. I've seen this done with C->LUA,
C->Python and Python->C, but there are certainly many more options.

--
Radomir `The Sheep' Dopieralski <http://sheep.art.pl>
Meden agan.
 >> Stay informed about: Language rewrite - any success stories?