I can be a very naive person too. To me, the most explanatory point is that a PhD is a Doctor of Philosophy, and it really means that, it’s not disconnected from its origin.
Academics are focused or publications and citations, and not other types of merits.This rewards people that want attention, not people that want to change the world. In theory, capitalism should do that (that’s another naive thought) so there may be good reasons why salaries in academia are not that impressive…
Tonysays:
I know what an academic is.
I don’t know what a hacker is. Is it someone who programs a computer? Or contributes to open-source projects? Or builds something, in general?
The word hacker is vague. This post seems to be saying: “Hackers, by definition, are those people that have positive qualities A, B, C, and D.”
Anonyymisays:
Here’s a problem: how can we foster the hacker spirit outside information technology? For some reason, only the programming community has realized the value of hackers. In physics, biology, medicine, etc. all of the little progress there is is still made by academics. Maybe we could indeed find a cure for cancer, room temperature superconductivity and quantum computing faster if instead of academics, the research was carried out by hackers.
Aramsays:
Surely you need both, and even some aspects of both should exist in anyone doing creative work. It is sometimes necessary to pursue a specific goal and to import/invent whatever techniques needed to achieve it. And sometimes one needs to synthesize, generalize, figure out the big picture, etc. It’s also sometimes useful to do these things without any specific application in mind.
Yes, academics sometimes get carried away by self-promotion, but for a theory to be successful, it should influence the work of others. Self-promotion, like marketing under capitalism, can sometimes be excessive and harmful, but still the socially optimal amount of it is nonzero.
At least, capitalism allows you try to build a cure for cancer if only you can convince investors… something that happens regularly (there are many start-ups with this business model).
Whether it rewards appropriately the inventors is another story.
nepiqensays:
Academics are shooting for more fundamental (i.e. broadly impactful) results than hacks. Sure, there’s a lot of self-promotion in academia, but mainly because very impactful results are few and far between, so it’s harder to judge what’s valuable. Yet, they are still worth pursuing because all the hacks in the world aren’t gonna get you warp drive – only a very basic fundamental change in the way we understand the world will get that. And yes, once that very fundamental understanding is in place, lots of practical little stuff needs to get done to realize a warp engine, but the fundamental understanding comes first.
So, it’s really a question of resource allocation. How much are you going to spend on fundamental research vs. engineering? You want to solve a problem, hacker? Come up with an algorithm to allocate resources for maximum output (whatever that is – you’ll have to define it). Ironically, if you get it, you’ll be considered a great scientist, not a hacker.
While practical engineering work is valuable, the reason a lot of people want to work on fundamental research is because the potential for changing the world is a lot bigger, if not probable. In comparison, hacks are boring to implement and not that impactful anymore. I really doubt that any hacker is going to create anything that’s going to save more than about 1% of my time anymore – in some cases (fb, anyone? endless perfunctory upgrades, anyone?), it may waste more time than it saves. The great strides in communication and information organization have pretty much already occurred. Yes, someday we will have something really super useful like genuine Turing test-passing AI, but that won’t come about quickly because of a clever hack – it’ll be more like a slow grinding approach.
Maybe big modern problems really do require more grinding, but let’s at least call it what it is.
Lorenzosays:
Daniel,
I’ve worked in academia but I consider myself more of a hacker. And I agree with your description so I wanted to weigh in my experience.
My primary concern with academia is that there was no culture of building a solid product; although I’m concerned mostly with code here I think what I’m saying could be generalised to other areas.
In my case I had to set up a set of good *practices* (not theories!) such as source control, testing, good design, etc. I insisted to hire a professional programmer to help write the code to run our simulations – a move frowned upon, although now there’s a nice spin-off with a good product that was good from the beginning and did not require rewrite.
Unfortunately if one wants to make a career in academia they don’t have to write good code. In fact, they shouldn’t waste time writing code at all – it’s best if it’s a proof of concept in Matlab to prove a new algorithm, and then ‘the industry will pick it up’. Nobody cares about performance, optimisation, and most importantly for quickly iterations.
In the end ‘quick iterations’ is what matter most in hacker culture: the ability to produce many prototypes, to iterate fast until a good solution is found, getting better and better every time. Academia discourages that because there is no time to finish anything; the next deadline is coming, then there’s teaching, managing, paperwork, and your (six month) contract is over thank you very much. How can one even get into the right state of mind to create something with a six month contract?
I don’t disagree with your points, but I think you left out the key weakness of the hacker approach to work: brittleness. Someone working in hacker mode can get some cool thing working in one particular context, but will it still be working tomorrow? Will it continue to work when someone else uses it in a different way? To expand on your own example, it’s great if you can work around a firewall, but can you do that because someone on reddit told you a weird vulcan nerve pinch for that product, or because you have a broad understanding of network principles and technologies?
At their best, academics provide the solid foundation of thoroughly analyzed theories, frameworks, etc that hackers can use (even if they don’t know it) to invent creatively. I don’t think the incentive systems in academia are very good at pushing academics to do this kind of work, but that’s a whole different can of worms.
Paulsays:
Speaking as a hacker, I like the system as it stands. Academics are encouraged to come up with good ideas, be it small incremental technique improvements or large isomorphisms in the world. They’re freed from the constraint of having to produce something useful, so they can, for example, spend a decade mining some small seam of knowledge.
As a hacker, I lack that particular luxury. I obtain a cursory understanding of a tool or problem, build something, and move to the next step. What lets me act so efficiently is that when I identify a product need, there’s often endless research available on the topic. I quickly sift through the chaff, find some smart ideas, and move forward. Not only can I find raw research, I can find survey papers and even proof-of-concept code or for-non-expert blog posts.
Hackers are bad at documentation. We’re bad at properly testing our systems against competing options. We often exist inside corporations where sharing knowledge outside is somewhat discouraged. I don’t view us as competing with academia, I view us having complementary roles.
mesays:
You of course need academic hackers.
That write good code, get others to contribute, and that can promote their code enough to be really used.
Otherwise, it is just another github zombie.
Not only can I find raw research, I can find survey papers and even proof-of-concept code or for-non-expert blog posts.
The overwhelming majority, over 99%, of all scientists will not provide you with proof of concepts. If they have code, they will not share it, even if you ask. They will not write blog posts for non experts. They will not give you freely their data.
I am not saying that they should, but it is not true that they do.
Hackers are bad at documentation.
How often have you tried to reproduce the results from a random research paper?
We often exist inside corporations where sharing knowledge outside is somewhat discouraged.
In the software industry, you can sit down today and watch YouTube videos of talks about how various things were built, until you are very, very tired of listening to all this “secret knowledge”.
In my experience, the very secretive bunch are the people who have nothing to show… they are scared that the truth be told.
Companies like Google tell us so much about the inner working of the company that it would be a full time job just to try to follow a faction of it all.
Heck! Even Goldman Sachs, the evil guys, publish their software libraries on GitHub.
There is a reason why this is happening… There is an invisible college. It is no accident that many of the great hackers know each other.
nepiqensays:
@Daniel
Ug. Now I remember why I stopped posting to comment sections. Why did I let my resolve fail?!?
I didn’t mean we need to understand everything before we can do anything (of course). Here are some examples of what I meant:
-Law of universal gravitation: 1690, actual space flight: wow. a lot later
-Maxwell equations: 1860s, applications: decades later
-General relativity: 1916, GPS (which depends on GR): 1990s
-Stimulated emission: ~1900s, lasers: 1950s
-QM:1920s, transistors, etc.: 1940s
-structure of DNA:1970s, lots of other stuff, decades later
-Early thermodynamics was developed along with early steam engine work, but these early steam engines might as well have been experiments given their actual practical impact on the world at the time (in the early 1800s).
And tons of other modern examples like NMR, where the theoretical basis was worked out way before medical imaging systems came along. As I wrote in my by you unappreciated original post: big discoveries are few and far between but research papers do indeed change the world. Check out the number of cites on the original NMR papers (or laser papers) if you doubt.
And here we are, not extinct!
The pretty little world of software engineering came about because very smart people solved problems which you, evidently, cannot actually conceive. If you want a taste of that world, write a device driver for buggy hardware with no specs, ’cause that’s the closest you’re gonna get. Oh, and then get somebody to fund you to do it 🙂 Nobody will, because writing software is only important until the next update two minutes later.
I loved coding when I was a kid, but it got boring, mainly because I couldn’t feel motivated to hack unless it was for a higher purpose (which is now the only reason I touch a computer).
But you are brilliant in your own way. With a very short, dumb post, you got somebody else to summarize what all citizens should understand about science when they are 12. Good show!
The pretty little world of software engineering came about because very smart people solved problems which you, evidently, cannot actually conceive.
Ad hominem.
Law of universal gravitation: 1690, actual space flight: wow. a lot later
Human beings dealt with gravitation in sophisticated ways centuries before 1690.
You do not honestly believe that the human race needed the Law of universal gravitation to cope with gravitation?
Maxwell equations: 1860s, applications: decades later
We had non-trivial industrial applications of electromagnetism decades before Maxwell equations. For example, the telegraph was invented in the 1830s, but the general idea had been toyed with a century earlier.
In fact, Maxwell provided a nice explanation of phenomenons that had been observed and put in practice for decades. Of course, it was a major contribution, but he certainly did not precede the applications.
Early thermodynamics was developed along with early steam engine work, but these early steam engines might as well have been experiments given their actual practical impact on the world at the time (in the early 1800s).
Thermodynamics came in the first half of the 1800s, after Watt had made a fortune commercializing the first engine, had a full life and died.
nepiqensays:
@Daniel
Weak.
You’re suggesting that a general theory of EM (or anything else) should predate an observation of (some of) the associated phenomena? That’s ridiculous. Do you actually expect me to provide yet another list of stuff that is not implied by the telegraph, but only came about after the Maxwell equations? I guess it doesn’t mean much that telegraph principles had been around (as you say) for 100 years before the MEs, but shortly after the MEs, radio magically appeared???
You simply can’t compare the practical impact made possible by the Maxwell equations to an early telegraph, same for thermodynamics and early steam engines, particularly if you include chemical engines. These applications are only impactful because the underlying phenomena, when fully understood, had absolutely huge consequences.
And are you seriously suggesting that we knew enough about gravity to enable space flight before Newton? You actually think that building arches compares to sending probes to Pluto???
And, heh, I notice you’ve got nothing to say about the other examples – I could easily dig into the lit. and find many more.
You’ve gotta be kidding me! You’re trolling your own message board!
You’re suggesting that a general theory of EM (or anything else) should predate an observation of (some of) the associated phenomena? That’s ridiculous.
Referring to upcoming inventions, you wrote that “the fundamental understanding comes first”. That’s the statement I object to.
History does not support this proposition. Someone first built an heavier-than-air flying machine, and then we got aerodynamics… someone first built an industrially useful engine, and then we got proper thermodynamics… we built a telegraph, and then we got proper electromagnetism…
We may very well build a warp drive before we understand the underlying physics.
And, heh, I notice you’ve got nothing to say about the other examples – I could easily dig into the lit. and find many more.
To contradict your statement, I only need to show that applications can precede understanding. I do not need to show that understanding may never lead to applications…
I am not saying that if you have a “warp theory”, you won’t get a warp drive. I am saying that you could get a warp drive before you have a warp theory.
The laser example you gave is a good one of understanding that precede applications. Laser was predicted to be possible, and it took industry engineering to build the first one. However, there are plenty of examples going in the other direction.
But this lead me to the really worrying thing… that the best way to make progress is to improve our understanding.
This approach, this “thinkism”, has routinely failed us. We diverted billions and decades for research time that would have been better spent solving real problems.
Studying problems only gets you so far.
nepiqensays:
@Daniel
Assuming that I meant no application related to EM could exist before EM itself is an overly semantical straw man. It is obvious to everyone that a thread leading to EM must exist prior to EM itself – again, why do I need to state this? It’s axiomatic, but does not demonstrate at all that the implications of EM would exist without EM itself.
All along you’ve advocated the practical route, yet here you resort to splitting philosophical hairs. The intuitively obvious fact is that the implications of EM are far greater than the implications of the few nuggets of knowledge that lead to the telegraph.
Continuing your metaphor: the telegraph is not warp theory, it is an application that is based on a nugget of practical information that was, necessarily, discovered first. EM is warp theory, and radio, fiber optics, and all digital computation are a warp engine.
You (finally) cite a couple examples of situations where practical applications were discovered “and then we got” the theory, but wrt heavier than air flight (for instance), you do realize that the equations of fluid flow had existed for decades when the Wright brothers flew. Bernoulli’s principle had been around for almost 200 years. The Wright brothers themselves had a process that was a whole lot more like experimental science than engineering. They built their own wind tunnel and specifically advanced instruments to measure quantities relevant to flight.
You say you have plenty of examples, but the one you cite above is just plain wrong, and you didn’t effectively refute any of mine. As far as I can tell, you are horrible at critical thinking. You criticize the formulation of powerful generalizations (which is the goal of science), but then you cavalierly fire one off yourself: Thinkism has routinely failed us??
What the hell is “thinkism?” Cite statistics where this well-defined property you’ve just pulled out of your butt has failed! Dude, it is clear that you are just grasping for whatever dignity you can save.
Ok, I’m done. I presume I’m going to get another load of rhetorical crap from you, so I’ll leave this last bit to other readers of this post (not you – please dear god don’t respond!):
Science is expensive. Its goal is to formulate rigorous generalizations of phenomena over a relatively large number of variables. The reason it’s expensive is because it covers a very large scope and experimental science (and life) is intrinsically uncertain. So, experimental scientists employ small scale incisive representations of the context of the phenomena of interest to reduce both the number of relevant variables and the uncertainty in their relationships. This process is inseparable from theoretical modeling based on experimental results, which is necessary to make predictions in alternate contexts, which can then be subjected to experiment to build confidence in the models.
Engineering applies the general, rigorous principles developed by science to implement practical applications on larger scales than science experiments *without uncertainty*. If engineering principles were applied without very general theoretical modeling, the resulting models would have very limited validity, and thus very limited application. I’d cite examples of failed projects based on this idea if I thought Dan would pay attention 🙂
Also, from a very fundamental point of view, pure engineering is unlikely to discover new phenomena, simply because it *does not search for them.*
The Wright brothers were high-school drop-outs who certainly did not lose sleep studying fluid mechanics, while those who did study fluid mechanics evidently could not design airplanes.
I did not finish reading your comment as you were being rude. I prefer when people stay clear of personal attacks.
Adamsays:
Just FYI, a lot of top conferences in the programing languages area adopted a process of “artifact evaluation” that encourages authors to submit “artifacts”, such as code and data used to generate numbers for the paper (which are then used by a separate committe to reproduce results presented in the paper), and rewards them for it. See for example http://conf.researchr.org/track/pldi2015/PLDI+2015+Artifact+Evaluation
There is even a list of Turing Award recipients which I would consider to be “hackers” in the noble sense of the term.
Computer science has been exemplary in this respect.
Adamsays:
@Daniel
And it’s not that I disagree with what you are saying, but the point that I was trying to make is that there are academics that see the problem and are actively trying to do something about it. A hacker academic (or academic hacker) is not s complete utopia! 🙂
The problem, as I see it, is that the bulk of the ressources, in academia, necessarily goes toward people who spend most of their time on promotion (grant applications, getting shiny papers). Helping people, curing diseases, solving real problems, is typically very secondary. Much better to work on nice problems that nobody has but that make nice papers… than to address reality…
Ulisays:
@Daniel:
I usually agree with your posts about 99%. With this one, however, I’m unsure if the number of percents will fit into a 32 bit integer 😉
I have seen the best of (or worst of) both worlds, at least to some extent. I have written a text-mining software in a research group until 2012. After that was finished successfully, I started to work on my own projects in a different way (you could call that a hacker).
3 years later, the academic software is now defunct. It was so complex that a full-time dedicated admin would be required in order to keep it running. The 20-server cluster has since been used for other projects (and I have reasons to believe that many if not most of them will face the same fate in the future). 0% of the software is open-source and anyone with access to the source code will need to work for months and months full-time to get it working — thousands of man-hours (estimated 10000) have basically been wasted with marginal results.
Since than I have published most of my own work as open source. While many projects are mostly uninteresting, I have received positive feedback.
Regarding the optional-fundamentals-before-progress thing I absolutely agree with you. For example, rocket science is not about understanding gravity. While being able to simulate gravity might get you to results faster, and understanding fuel flow in engines might enable you to build better engines with trial-and-error, people (including the early Chinese, if I recall correctly) have built rockets without any knowledge of the law of gravitation or relativity.
Try talking to a theoretical physicist, let’s see if he can setup a GPS duplicate. 😉
For me, science — IF done correctly — is an acceleration. As an alternative to this process, I think massively parallel trial and error makes sense. To cite Torvalds: “And don’t ever make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle”.
The crux is, that IMO science is mostly done with maximum incorrectness. Even really honest scientists who would like to advance their fields often have little success other than producing tons of paper(s) without lasting impact.
Two examples:
a) I have heard multiple times that people who actually generated results needed to split their work into multiple (sometimes tens of) papers so 1) they might get a larger impact factor and 2) it is more “safe” in the sense that the likelihood for a rejected paper decreases if 99% of your paper is already known
b) Someone that oversees people who are trying to get a biology doctorate told me last year that quite often the doctorates waste thousands of man-hours (equating to 20-70% of their entire doctorate time) 1) trying to replicate results without success and 2) doing experiments that generations of biologists did not succeed in, but no-one published as negative results are not publishable in the reputable journals.
Fortunately there are exceptions, but as the citizens pay science via taxes, I believe that one needs to demand that transparency in terms of open-source and open-data must not be the exception but the default.
Yes, some science is expensive. Therefore I think we need to make sure that the incredible amount of money that is (and to some extent needs to be) spent is used in a constructive way.
Although I have the highest respect for people trying to do science the idealistic way and for the idealistic concept of science itself, what I observe is vastly different from that.
I don’t think there’s a single reason to blame for that, but “the system”, including work contracts, wages, reputation by papers and impact factors and last but not least some professors who will declare any hacker-ish work as too “unclean” or “unpure” for their subject and therefore will prevent most progress for the decades to follow (because their successors are basically selected by themselves).
Academic hacker (biased?)says:
Let aside the matter of definitions, which are necessary, I prefer to move into more core issues regarding your argument. Moreover, I won’t open a discussion on possible logical fallacies or even the initial placement of the question (does it have to be one of the two?).
So the main issue/problem regarding your argument is bias. It seems that you related to some hacking background develop bias (initially shown from the question) which does not allow you to see further.
This leads in comparing a theoretical or rather romantic view of hacker to a really bad example of an academic.
Having said that, allow me to post my view. Well, I believe that anyone can be responsible for progress, thus no distinctions based on somekind of label. Secondly, in my opinion hackers and academics can be quite alike. I believe this happens because both have pretty common basis, curiosity, like to study, analyse and imagine, create new knowledge. And believe me they both have passion for it. If you are familiar with “Science is like sex. ” quote Richard Feynman.
After all many hackers are/were academics and vice versa.
May I place a question which might make things more clear, “Who came first, the hacker or the academic” and not hacker just as IT guy but in general hacker.
>That is the primary purpose of research grants: hire more helping hands.
Hire more hackers!
I can be a very naive person too. To me, the most explanatory point is that a PhD is a Doctor of Philosophy, and it really means that, it’s not disconnected from its origin.
Academics are focused or publications and citations, and not other types of merits.This rewards people that want attention, not people that want to change the world. In theory, capitalism should do that (that’s another naive thought) so there may be good reasons why salaries in academia are not that impressive…
I know what an academic is.
I don’t know what a hacker is. Is it someone who programs a computer? Or contributes to open-source projects? Or builds something, in general?
The word hacker is vague. This post seems to be saying: “Hackers, by definition, are those people that have positive qualities A, B, C, and D.”
Here’s a problem: how can we foster the hacker spirit outside information technology? For some reason, only the programming community has realized the value of hackers. In physics, biology, medicine, etc. all of the little progress there is is still made by academics. Maybe we could indeed find a cure for cancer, room temperature superconductivity and quantum computing faster if instead of academics, the research was carried out by hackers.
Surely you need both, and even some aspects of both should exist in anyone doing creative work. It is sometimes necessary to pursue a specific goal and to import/invent whatever techniques needed to achieve it. And sometimes one needs to synthesize, generalize, figure out the big picture, etc. It’s also sometimes useful to do these things without any specific application in mind.
Yes, academics sometimes get carried away by self-promotion, but for a theory to be successful, it should influence the work of others. Self-promotion, like marketing under capitalism, can sometimes be excessive and harmful, but still the socially optimal amount of it is nonzero.
@Anonyymi
Have you googled “biohacker” recently?
@trylks
At least, capitalism allows you try to build a cure for cancer if only you can convince investors… something that happens regularly (there are many start-ups with this business model).
Whether it rewards appropriately the inventors is another story.
Academics are shooting for more fundamental (i.e. broadly impactful) results than hacks. Sure, there’s a lot of self-promotion in academia, but mainly because very impactful results are few and far between, so it’s harder to judge what’s valuable. Yet, they are still worth pursuing because all the hacks in the world aren’t gonna get you warp drive – only a very basic fundamental change in the way we understand the world will get that. And yes, once that very fundamental understanding is in place, lots of practical little stuff needs to get done to realize a warp engine, but the fundamental understanding comes first.
So, it’s really a question of resource allocation. How much are you going to spend on fundamental research vs. engineering? You want to solve a problem, hacker? Come up with an algorithm to allocate resources for maximum output (whatever that is – you’ll have to define it). Ironically, if you get it, you’ll be considered a great scientist, not a hacker.
While practical engineering work is valuable, the reason a lot of people want to work on fundamental research is because the potential for changing the world is a lot bigger, if not probable. In comparison, hacks are boring to implement and not that impactful anymore. I really doubt that any hacker is going to create anything that’s going to save more than about 1% of my time anymore – in some cases (fb, anyone? endless perfunctory upgrades, anyone?), it may waste more time than it saves. The great strides in communication and information organization have pretty much already occurred. Yes, someday we will have something really super useful like genuine Turing test-passing AI, but that won’t come about quickly because of a clever hack – it’ll be more like a slow grinding approach.
Maybe big modern problems really do require more grinding, but let’s at least call it what it is.
Daniel,
I’ve worked in academia but I consider myself more of a hacker. And I agree with your description so I wanted to weigh in my experience.
My primary concern with academia is that there was no culture of building a solid product; although I’m concerned mostly with code here I think what I’m saying could be generalised to other areas.
In my case I had to set up a set of good *practices* (not theories!) such as source control, testing, good design, etc. I insisted to hire a professional programmer to help write the code to run our simulations – a move frowned upon, although now there’s a nice spin-off with a good product that was good from the beginning and did not require rewrite.
Unfortunately if one wants to make a career in academia they don’t have to write good code. In fact, they shouldn’t waste time writing code at all – it’s best if it’s a proof of concept in Matlab to prove a new algorithm, and then ‘the industry will pick it up’. Nobody cares about performance, optimisation, and most importantly for quickly iterations.
In the end ‘quick iterations’ is what matter most in hacker culture: the ability to produce many prototypes, to iterate fast until a good solution is found, getting better and better every time. Academia discourages that because there is no time to finish anything; the next deadline is coming, then there’s teaching, managing, paperwork, and your (six month) contract is over thank you very much. How can one even get into the right state of mind to create something with a six month contract?
One last note – there’s an interesting, albeit very long, article at https://michaelochurch.wordpress.com/2015/09/04/how-software-engineers-failed-to-get-the-status-they-deserve/ that captures very well why “hackers” don’t play politics, and why is this clashing with the venture capitalist culture of late.
I don’t disagree with your points, but I think you left out the key weakness of the hacker approach to work: brittleness. Someone working in hacker mode can get some cool thing working in one particular context, but will it still be working tomorrow? Will it continue to work when someone else uses it in a different way? To expand on your own example, it’s great if you can work around a firewall, but can you do that because someone on reddit told you a weird vulcan nerve pinch for that product, or because you have a broad understanding of network principles and technologies?
At their best, academics provide the solid foundation of thoroughly analyzed theories, frameworks, etc that hackers can use (even if they don’t know it) to invent creatively. I don’t think the incentive systems in academia are very good at pushing academics to do this kind of work, but that’s a whole different can of worms.
Speaking as a hacker, I like the system as it stands. Academics are encouraged to come up with good ideas, be it small incremental technique improvements or large isomorphisms in the world. They’re freed from the constraint of having to produce something useful, so they can, for example, spend a decade mining some small seam of knowledge.
As a hacker, I lack that particular luxury. I obtain a cursory understanding of a tool or problem, build something, and move to the next step. What lets me act so efficiently is that when I identify a product need, there’s often endless research available on the topic. I quickly sift through the chaff, find some smart ideas, and move forward. Not only can I find raw research, I can find survey papers and even proof-of-concept code or for-non-expert blog posts.
Hackers are bad at documentation. We’re bad at properly testing our systems against competing options. We often exist inside corporations where sharing knowledge outside is somewhat discouraged. I don’t view us as competing with academia, I view us having complementary roles.
You of course need academic hackers.
That write good code, get others to contribute, and that can promote their code enough to be really used.
Otherwise, it is just another github zombie.
@nepiqen
lots of practical little stuff needs to get done to realize a warp engine, but the fundamental understanding comes first.
If we needed to understand the problems before we solved them, the human race would have gone extinct a long time ago.
a lot of people want to work on fundamental research is because the potential for changing the world is a lot bigger
Changing the world by writing research papers?
I think people write research papers for prestige mostly. Changing the world is very far down the list of motivations.
@Paul
Not only can I find raw research, I can find survey papers and even proof-of-concept code or for-non-expert blog posts.
The overwhelming majority, over 99%, of all scientists will not provide you with proof of concepts. If they have code, they will not share it, even if you ask. They will not write blog posts for non experts. They will not give you freely their data.
I am not saying that they should, but it is not true that they do.
Hackers are bad at documentation.
How often have you tried to reproduce the results from a random research paper?
We often exist inside corporations where sharing knowledge outside is somewhat discouraged.
In the software industry, you can sit down today and watch YouTube videos of talks about how various things were built, until you are very, very tired of listening to all this “secret knowledge”.
In my experience, the very secretive bunch are the people who have nothing to show… they are scared that the truth be told.
Companies like Google tell us so much about the inner working of the company that it would be a full time job just to try to follow a faction of it all.
Heck! Even Goldman Sachs, the evil guys, publish their software libraries on GitHub.
There is a reason why this is happening… There is an invisible college. It is no accident that many of the great hackers know each other.
@Daniel
Ug. Now I remember why I stopped posting to comment sections. Why did I let my resolve fail?!?
I didn’t mean we need to understand everything before we can do anything (of course). Here are some examples of what I meant:
-Law of universal gravitation: 1690, actual space flight: wow. a lot later
-Maxwell equations: 1860s, applications: decades later
-General relativity: 1916, GPS (which depends on GR): 1990s
-Stimulated emission: ~1900s, lasers: 1950s
-QM:1920s, transistors, etc.: 1940s
-structure of DNA:1970s, lots of other stuff, decades later
-Early thermodynamics was developed along with early steam engine work, but these early steam engines might as well have been experiments given their actual practical impact on the world at the time (in the early 1800s).
And tons of other modern examples like NMR, where the theoretical basis was worked out way before medical imaging systems came along. As I wrote in my by you unappreciated original post: big discoveries are few and far between but research papers do indeed change the world. Check out the number of cites on the original NMR papers (or laser papers) if you doubt.
And here we are, not extinct!
The pretty little world of software engineering came about because very smart people solved problems which you, evidently, cannot actually conceive. If you want a taste of that world, write a device driver for buggy hardware with no specs, ’cause that’s the closest you’re gonna get. Oh, and then get somebody to fund you to do it 🙂 Nobody will, because writing software is only important until the next update two minutes later.
I loved coding when I was a kid, but it got boring, mainly because I couldn’t feel motivated to hack unless it was for a higher purpose (which is now the only reason I touch a computer).
But you are brilliant in your own way. With a very short, dumb post, you got somebody else to summarize what all citizens should understand about science when they are 12. Good show!
@nepiqen
The pretty little world of software engineering came about because very smart people solved problems which you, evidently, cannot actually conceive.
Ad hominem.
Law of universal gravitation: 1690, actual space flight: wow. a lot later
Human beings dealt with gravitation in sophisticated ways centuries before 1690.
You do not honestly believe that the human race needed the Law of universal gravitation to cope with gravitation?
Maxwell equations: 1860s, applications: decades later
We had non-trivial industrial applications of electromagnetism decades before Maxwell equations. For example, the telegraph was invented in the 1830s, but the general idea had been toyed with a century earlier.
In fact, Maxwell provided a nice explanation of phenomenons that had been observed and put in practice for decades. Of course, it was a major contribution, but he certainly did not precede the applications.
Early thermodynamics was developed along with early steam engine work, but these early steam engines might as well have been experiments given their actual practical impact on the world at the time (in the early 1800s).
Thermodynamics came in the first half of the 1800s, after Watt had made a fortune commercializing the first engine, had a full life and died.
@Daniel
Weak.
You’re suggesting that a general theory of EM (or anything else) should predate an observation of (some of) the associated phenomena? That’s ridiculous. Do you actually expect me to provide yet another list of stuff that is not implied by the telegraph, but only came about after the Maxwell equations? I guess it doesn’t mean much that telegraph principles had been around (as you say) for 100 years before the MEs, but shortly after the MEs, radio magically appeared???
You simply can’t compare the practical impact made possible by the Maxwell equations to an early telegraph, same for thermodynamics and early steam engines, particularly if you include chemical engines. These applications are only impactful because the underlying phenomena, when fully understood, had absolutely huge consequences.
And are you seriously suggesting that we knew enough about gravity to enable space flight before Newton? You actually think that building arches compares to sending probes to Pluto???
And, heh, I notice you’ve got nothing to say about the other examples – I could easily dig into the lit. and find many more.
You’ve gotta be kidding me! You’re trolling your own message board!
@nepiqen
You’re suggesting that a general theory of EM (or anything else) should predate an observation of (some of) the associated phenomena? That’s ridiculous.
Referring to upcoming inventions, you wrote that “the fundamental understanding comes first”. That’s the statement I object to.
History does not support this proposition. Someone first built an heavier-than-air flying machine, and then we got aerodynamics… someone first built an industrially useful engine, and then we got proper thermodynamics… we built a telegraph, and then we got proper electromagnetism…
We may very well build a warp drive before we understand the underlying physics.
And, heh, I notice you’ve got nothing to say about the other examples – I could easily dig into the lit. and find many more.
To contradict your statement, I only need to show that applications can precede understanding. I do not need to show that understanding may never lead to applications…
I am not saying that if you have a “warp theory”, you won’t get a warp drive. I am saying that you could get a warp drive before you have a warp theory.
The laser example you gave is a good one of understanding that precede applications. Laser was predicted to be possible, and it took industry engineering to build the first one. However, there are plenty of examples going in the other direction.
But this lead me to the really worrying thing… that the best way to make progress is to improve our understanding.
This approach, this “thinkism”, has routinely failed us. We diverted billions and decades for research time that would have been better spent solving real problems.
Studying problems only gets you so far.
@Daniel
Assuming that I meant no application related to EM could exist before EM itself is an overly semantical straw man. It is obvious to everyone that a thread leading to EM must exist prior to EM itself – again, why do I need to state this? It’s axiomatic, but does not demonstrate at all that the implications of EM would exist without EM itself.
All along you’ve advocated the practical route, yet here you resort to splitting philosophical hairs. The intuitively obvious fact is that the implications of EM are far greater than the implications of the few nuggets of knowledge that lead to the telegraph.
Continuing your metaphor: the telegraph is not warp theory, it is an application that is based on a nugget of practical information that was, necessarily, discovered first. EM is warp theory, and radio, fiber optics, and all digital computation are a warp engine.
You (finally) cite a couple examples of situations where practical applications were discovered “and then we got” the theory, but wrt heavier than air flight (for instance), you do realize that the equations of fluid flow had existed for decades when the Wright brothers flew. Bernoulli’s principle had been around for almost 200 years. The Wright brothers themselves had a process that was a whole lot more like experimental science than engineering. They built their own wind tunnel and specifically advanced instruments to measure quantities relevant to flight.
You say you have plenty of examples, but the one you cite above is just plain wrong, and you didn’t effectively refute any of mine. As far as I can tell, you are horrible at critical thinking. You criticize the formulation of powerful generalizations (which is the goal of science), but then you cavalierly fire one off yourself: Thinkism has routinely failed us??
What the hell is “thinkism?” Cite statistics where this well-defined property you’ve just pulled out of your butt has failed! Dude, it is clear that you are just grasping for whatever dignity you can save.
Ok, I’m done. I presume I’m going to get another load of rhetorical crap from you, so I’ll leave this last bit to other readers of this post (not you – please dear god don’t respond!):
Science is expensive. Its goal is to formulate rigorous generalizations of phenomena over a relatively large number of variables. The reason it’s expensive is because it covers a very large scope and experimental science (and life) is intrinsically uncertain. So, experimental scientists employ small scale incisive representations of the context of the phenomena of interest to reduce both the number of relevant variables and the uncertainty in their relationships. This process is inseparable from theoretical modeling based on experimental results, which is necessary to make predictions in alternate contexts, which can then be subjected to experiment to build confidence in the models.
Engineering applies the general, rigorous principles developed by science to implement practical applications on larger scales than science experiments *without uncertainty*. If engineering principles were applied without very general theoretical modeling, the resulting models would have very limited validity, and thus very limited application. I’d cite examples of failed projects based on this idea if I thought Dan would pay attention 🙂
Also, from a very fundamental point of view, pure engineering is unlikely to discover new phenomena, simply because it *does not search for them.*
Git ‘er done!
@nepiqen
The Wright brothers were high-school drop-outs who certainly did not lose sleep studying fluid mechanics, while those who did study fluid mechanics evidently could not design airplanes.
I did not finish reading your comment as you were being rude. I prefer when people stay clear of personal attacks.
Just FYI, a lot of top conferences in the programing languages area adopted a process of “artifact evaluation” that encourages authors to submit “artifacts”, such as code and data used to generate numbers for the paper (which are then used by a separate committe to reproduce results presented in the paper), and rewards them for it. See for example http://conf.researchr.org/track/pldi2015/PLDI+2015+Artifact+Evaluation
@Adam
Sure.
There is even a list of Turing Award recipients which I would consider to be “hackers” in the noble sense of the term.
Computer science has been exemplary in this respect.
@Daniel
And it’s not that I disagree with what you are saying, but the point that I was trying to make is that there are academics that see the problem and are actively trying to do something about it. A hacker academic (or academic hacker) is not s complete utopia! 🙂
@Adam
The problem, as I see it, is that the bulk of the ressources, in academia, necessarily goes toward people who spend most of their time on promotion (grant applications, getting shiny papers). Helping people, curing diseases, solving real problems, is typically very secondary. Much better to work on nice problems that nobody has but that make nice papers… than to address reality…
@Daniel:
I usually agree with your posts about 99%. With this one, however, I’m unsure if the number of percents will fit into a 32 bit integer 😉
I have seen the best of (or worst of) both worlds, at least to some extent. I have written a text-mining software in a research group until 2012. After that was finished successfully, I started to work on my own projects in a different way (you could call that a hacker).
3 years later, the academic software is now defunct. It was so complex that a full-time dedicated admin would be required in order to keep it running. The 20-server cluster has since been used for other projects (and I have reasons to believe that many if not most of them will face the same fate in the future). 0% of the software is open-source and anyone with access to the source code will need to work for months and months full-time to get it working — thousands of man-hours (estimated 10000) have basically been wasted with marginal results.
Since than I have published most of my own work as open source. While many projects are mostly uninteresting, I have received positive feedback.
Regarding the optional-fundamentals-before-progress thing I absolutely agree with you. For example, rocket science is not about understanding gravity. While being able to simulate gravity might get you to results faster, and understanding fuel flow in engines might enable you to build better engines with trial-and-error, people (including the early Chinese, if I recall correctly) have built rockets without any knowledge of the law of gravitation or relativity.
Try talking to a theoretical physicist, let’s see if he can setup a GPS duplicate. 😉
For me, science — IF done correctly — is an acceleration. As an alternative to this process, I think massively parallel trial and error makes sense. To cite Torvalds: “And don’t ever make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle”.
The crux is, that IMO science is mostly done with maximum incorrectness. Even really honest scientists who would like to advance their fields often have little success other than producing tons of paper(s) without lasting impact.
Two examples:
a) I have heard multiple times that people who actually generated results needed to split their work into multiple (sometimes tens of) papers so 1) they might get a larger impact factor and 2) it is more “safe” in the sense that the likelihood for a rejected paper decreases if 99% of your paper is already known
b) Someone that oversees people who are trying to get a biology doctorate told me last year that quite often the doctorates waste thousands of man-hours (equating to 20-70% of their entire doctorate time) 1) trying to replicate results without success and 2) doing experiments that generations of biologists did not succeed in, but no-one published as negative results are not publishable in the reputable journals.
Fortunately there are exceptions, but as the citizens pay science via taxes, I believe that one needs to demand that transparency in terms of open-source and open-data must not be the exception but the default.
Yes, some science is expensive. Therefore I think we need to make sure that the incredible amount of money that is (and to some extent needs to be) spent is used in a constructive way.
Although I have the highest respect for people trying to do science the idealistic way and for the idealistic concept of science itself, what I observe is vastly different from that.
I don’t think there’s a single reason to blame for that, but “the system”, including work contracts, wages, reputation by papers and impact factors and last but not least some professors who will declare any hacker-ish work as too “unclean” or “unpure” for their subject and therefore will prevent most progress for the decades to follow (because their successors are basically selected by themselves).
Let aside the matter of definitions, which are necessary, I prefer to move into more core issues regarding your argument. Moreover, I won’t open a discussion on possible logical fallacies or even the initial placement of the question (does it have to be one of the two?).
So the main issue/problem regarding your argument is bias. It seems that you related to some hacking background develop bias (initially shown from the question) which does not allow you to see further.
This leads in comparing a theoretical or rather romantic view of hacker to a really bad example of an academic.
Having said that, allow me to post my view. Well, I believe that anyone can be responsible for progress, thus no distinctions based on somekind of label. Secondly, in my opinion hackers and academics can be quite alike. I believe this happens because both have pretty common basis, curiosity, like to study, analyse and imagine, create new knowledge. And believe me they both have passion for it. If you are familiar with “Science is like sex. ” quote Richard Feynman.
After all many hackers are/were academics and vice versa.
May I place a question which might make things more clear, “Who came first, the hacker or the academic” and not hacker just as IT guy but in general hacker.
P.s. It’s a fallacy…