Stephan Paukner :: syslog

Ein Nachtrag fürs Protokoll: Ich bin seit Jahresanfang auf GigaSpeed 30 (30 Mbps Downstream/4 Mbps Up) der A1 Telekom, deren Produktreihe sich mittlerweile auf Glasfaser Speed Power umbenannt hat. Hauptgrund für das „vorgezogene“ Upgrade vom 16er-Blech äh Paket (16 Down/1 Up) war der Upstream, der für mich angesichts des langsamen Einschleichens von Cloud-Speichern in den elektronischen Alltag relevant geworden war.

Mit dem Preisverfall deren 30er-Pakets von 14,90 auf 9,90 €/M Mitte 2012 war plötzlich die Verfügbarkeit an meinem ländlichen Wohnort nicht mehr gegeben, obwohl das Paket mir offensichtlich vor drei Jahren zur Auswahl stand. Ich lauerte in den Monaten danach auf die Wiederverfügbarkeit, und tatsächlich, im November tauchte es wieder auf. Der Bestellvorgang ermittelte dann allerdings doch Nichtverfügbarkeit. Erst im Februar war mir die tatsächliche Bestellung möglich.

Das Modem hat sich auf 22,5 Mbps Downstream und 3,1 Mbps Upstream eingependelt. Die Gesamtkosten belaufen sich übrigens auf annehmbare 31,05 €/M (mit 19,90 für das Basispaket und €15÷12 Pauschale).

In gewissem Sinne sind das „drei Schritte vor, zwei zurück“: Zwar steigerte sich der Upstream von 1 auf 3 Mbps, doch werden aufgrund demnächst verdoppelter Pixelzahl meine RAW-Fotos doppelt so groß.

It took me some time to write a post about photo gear.

As my Canon EOS 40D is now really becoming old, I’m looking for an upgrade. Sadly, my photography hasn’t evolved much, so I most probably won’t wait for the 7D Mk II (if that one still makes sense, given the 6D) and will stay with their prosumer x0D line. Surprisingly, it took Canon three years to introduce a successor to the 60D: The EOS 70D. Although I originally considered the 60D a downgrade within that line, my view has changed with its successor: The reduced weight is not something negative, the reduced size is minimal, the plastic housing at least has got weather sealing, and the swivel screen perfectly fits the camera’s video capabilities. I’d get (compared to the 40D):

• 20 MP instead of 10 MP,
• DIGIC 5+ processor instead of DIGIC 3,
• ISO 12800 max. instead of ISO 1600,
• 19 AF points instead of 9,
• Dual-pixel AF in Live View instead of a manual trigger with mirror flip,
• 98% viewfinder coverage instead of 95%,
• [Update: Viewfinder with switchable gridline overlay and electronic level,]
  
• HD movies instead of no movies at all,
• HDMI output instead of plain old analog,
• Weather sealing instead of none,
• High-res swivel touch screen instead of low-res static plain LCD,
• Built-in Wi-Fi for remote operation using Canon’s smartphone app(!!),
• Built-in flash as off-camera remote flash controller!
• Continuous shooting with a rate of 7 fps instead of 6.5 fps, but the RAW image burst goes down from 17 to 16 and the JPEG burst from 75 to 65.

So, quite a jump in technology during the last six years. I plan my purchase for the end of this year.

Some quotes of myself from four years ago:

“Cheap” should mean here that you don’t need to spend money on extra hardware like a remote timer or on extra software like Windoze [...]

My hope is that I can use gphoto2 with an Android smartphone.

Somehow this is still a Linux-post, as Android simply is the most popular Linux distribution to date. Btw, PC and smartphone don’t count as dedicated extra hardware. But you might need to purchase an app and a USB adapter:

Meanwhile I use the awesome DSLR Controller on my Android smartphone in USB host-mode to create time-lapse picture sequences on my aging Canon EOS 40D—a thing that wouldn’t work with a diePhone, I guess. This app can do a lot more, of course. Up to now, you have to rely on USB. However, as Wi-Fi enabled DSLRs are appearing, there is some hope to get rid of that cable and adapter one day; Canon’s own EOS Remote app doesn’t support time-lapse shooting yet. [Update: To equip your camera with Wi-Fi, you could hack an Android TV stick!]

The most elegant solution would of course be located within the camera itself. Canon is still sleeping in this regard, but at least there’s the third-party Magic Lantern firmware add-on for some newer models, also sporting an intervalometer. My 40D is not (yet, but I guess won’t ever be) supported.

You should still set the camera to a resolution at or slightly above the HD 1080p resolution, which in my case is 1936×1288. You should fix the aperture and ISO values, and probably also the exposure time. [Update: In addition, fix the white balance. Also, don’t forget to cover the viewfinder with the eyepiece cover on your strap, as otherwise you might get different exposures that result in flickering!]

After getting the images to your Linux machine, you need to crop the pictures from 3:2 to 16:9 (in my case 1936×1089) or crop a 1920×1080 patch directly. You can do this with a simple script using ImageMagick:

(GNU Parallel didn’t work for me.) You can then issue

to render an HD video into a format that’s also recognized by your smartphone.

Finally, you could use OpenShot to edit your videos and add background music. (You could of course compose that music yourself using FOSS as well...)

I started the following section as a draft in 2008, with the generic title “Mobile device wanted”:

• I need an electronic device with about the shape and weight of a usual magazine or book.
• It should have an electronic touch-screen with a reasonably high resolution, in a size that covers most of the front area.
• The device should be capable of storing files of all type.
• The device’s software should be capable of:
• Showing documents of various formats (especially PDF) that can be scrolled by touching the screen.
• The screen should capture and visualize touch-pen movements correspondingly, e.g. to underline phrases or add hand-written notes. (Most important!)
• Bookmark functionality and cross-document references. (OK, just nice to have.)
• These notes should be stored in some way, either directly in the document (if the format allows it), or in a separate database or file that maybe even allows exporting the additional information.
• Editing editable document formats such as DOC or ODT by translating touch-pen movements to text is optional.
• Optional features:
• Keyboard underneath or attachable.
• Network connection (LAN/Wi-Fi) and internet/e-mail/WWW software.

I want such a device as I read many scientific papers/PDFs for work. It’s not convenient to print them all out, gather them as a bunch of single sheets and take notes with a real pen while e.g. on a train. It would be nice to have a PC-like device, as software and operating systems already exist, and it would immediately be possible to search the web for further information.

It seems that such a device is finally coming in two different incarnations: As netbook or as e-reader.

Yes. The ¡Pad didn’t exist yet. That mentioned netbook back then was a convertible, with a touch-screen LCD that could be rotated to be operated like a tablet. Similar solutions still exist, e.g. the Lenovo ThinkPad X230, but I dislike the low screen resolution, its weight and the MS Windows 7 OS that’s not tailored for touch use. But the main reason why I disliked both the netbook and the e-readers was that these were no dedicated solutions for stylus input—I wanted to write formulas and draw graphs and arrows. I followed the evolution of e-readers with e-ink displays, but they were just too laggy.

When the ¡Pad was introduced in 2010, I saw the desired technology approaching, but I was leering at a more open Android solution. Sadly, in the first time those tablets were only targeted at game and movie enthusiasts, and were more considered gadgets than productive devices. I was drooling over Plastic Logic’s QUE e-reader that also had a stylus, but of course it never appeared on the market.

The finally tablet-optimized Android 3 of 2011 wasn’t mature. Also, I still think that 10” screen diagonals are somewhat small compared to textbooks or magazines, so I was drooling over Kno’s 14” tablet. (I shake my head over opinions that 10” are already too large.) Of course, Kno went out of the hardware business before reaching production. From a different manufacturer, and while too underweight for my needs, the NoteSlate was a nice concept, but it still doesn’t exist.

2011 also brought Samsung’s S Pen technology with their Android-based Note brand, and I thought that a tablet variant would be just around the corner. Indeed, the Note 10.1 came in 2012, but it still had that pixelated 720p resolution (regarding font rendering) instead of more reasonable 1080p. Finally, there seem to be two 11” Samsung tablets lurking for 2013, but none of them is Note branded; there’s also a rumor that there won’t be a Note 10.1 successor soon due to weak sales of the current model.

It appears I’ll have to wait for yet another year.

From: me
To: info@[a technical job board].com
Subject: Re: Please delete my account

[a technical job board] wrote:

Your account has been deleted. Please inform us about the reason you want to close your account.
We would like to improve our services and the website all the time so I hope you can give me some valuable feedback.

Reason for deletion: I was shocked to see that after registration, I was sent my password in an email in clear text.
There are two things wrong with that:
1. You obviously store users’ passwords in clear text.
2. You send passwords via email, an inherently insecure protocol.
While your service might be intended for technically versed professionals, these facts state that your service isn’t run by such.

• A Strategic Mistake With Big Data — International Institute for Analytics

  tags: bigdata opinion

  • Bill Franks
  • That mistake is the development of a siloed, distinct big data strategy
  • strategy for big data is a new facet of their overall enterprise data and analytic strategy
  • the mess that many multi-channel retailers got themselves into through their entry into e-commerce
  • retailers launched distinct e-commerce divisions. Some were even separate legal entities. As opposed to viewing e-commerce as a new facet of an overall retail strategy
  • distinct processes and distinct infrastructure was created
  • provide a consistent experience for customers across channels
  • go into a store and grab a product and then find that same product on the retailer’s website.  Guess what? They have no way to match those products
  • separate, non-integrated strategies for big data will likely end up with systems and processes that are very difficult to integrate
  • integrate big data into the overall infrastructure and current and future analytic processes

• Machine learning system can ID cities via pics | Cutting Edge - CNET News

  tags: machinelearning computervision datamining research technology

  • The system automatically picks out relevant architectural details from photos
  • the details woven into the urban fabric that form a pattern
  • computers are learning to ID your city just by looking at random photos
  • Google Street View images of Paris, London, New York, and Barcelona
  • features like the street signs, balconies, and lampposts of Paris to be distinct
  • lack of stylistic coherence in American cities
  • presented at Siggraph 2012
  • emerging field of visual data mining, which is more complex than looking for patterns in text or numbers
  • Alexei Efros
  • we wish to automatically build a digital visual atlas of not only architectural but also natural geo-informative features for the entire planet

• Interviewing data scientist candidates? Ask these questions

  tags: datascience career opinion

  • “It’s way over-hyped,” said Franks
  • chief analytics officer for the data warehouse appliance vendor Teradata
  • “softer skills.”
  • “data artist.”
  • data scientists are those that are able to understand the business problem; they’re able to apply creativity and present the results well
  • intuition, which is hard to teach
  • five core areas I look at
  • Commitment
    • 1.
  • Creativity
    • 2.
  • thought process they went through deciding what to do. Someone who’s not creative is going to give me a list of the steps they went through one-by-one more from a technical perspective
  • Business savvy
    • 3.
  • what I want to hear is not just some technical reasons
  • Knowing how much information to give to the non-technical people
  • Presentation
    • 4.
  • specific presentation as part of the interview process
  • intuition
    • 5.
  • art or music or some other type of creative area

• Linux lessons for Hadoop doubters • The Register

  Interesting comparison of Hadoop today with the Linux story from the past. This could mean Hadoop/MapReduce as state of the art around 2020.

  tags: bigdata cloud technology linux

  • While Hadoop is all the rage in the technology media today, it has barely scratched the surface of enterprise adoption
  • Hadoop seems set to win despite its many shortcomings
  • still in the transition from zero per cent adoption to one per cent adoption
  • IBM points to a few specific deficiencies
  • lack of performance and scalability, inflexible resource management, and a limitation to a single distributed file system
  • IBM, of course, promises to resolve these issues with its proprietary complements to Hadoop
  • Hadoop is batch oriented in a world increasingly run in real-time
  • customers are buying big into Hadoop
  • it’s still possible that other alternatives, like Percolator, will claim the Hadoop crown
  • Back in 2000 IBM announced that it was going to invest $1bn in advancing the Linux operating system. This was big news
  • it came roughly 10 years after Linus Torvalds released the first Linux source code, and it took another 10 years before Linux really came to dominate the industry
  • The same seems true of Hadoop today
  • we’re just starting the marathon

• Can Big Data cut through your growing resume pile? - Fortune Management

  Three data-mangling job sites, all only for the US

  tags: bigdata career

  • Bright, one of several new companies
  • another new job site, Path.to
  • Gild, a third major player

• How ‘Big Data’ Is Different

  tags: bigdata datascience technology opinion

  • Thomas H. Davenport, Paul Barth and Randy Bean
  • how do the potential insights from big data differ from what managers generate from traditional analytics?
  • 1. Paying attention to flows as opposed to stocks
  • the data is not the “stock” in a data warehouse but a continuous flow
  • Streaming analytics
  • organizations will need to develop continuous processes
  • data extraction, preparation and analysis took weeks to prepare — and weeks more to execute
  • conventional, high-certitude approaches to decision-making are often not appropriate
  • new data is often available that renders the decision obsolete
  • 2. Relying on data scientists and product and process developers as opposed to data analysts
  • the people who work with big data need substantial and creative IT skills
  • programming, mathematical and statistical skills, as well as business acumen and the ability to communicate effectively
  • EMC Corporation
  • started an educational offering for data scientists
  • 3. Moving analytics from IT into core business and operational functions
  • new products designed to deal with big data
  • Hadoop
  • Relational databases have also been transformed
  • Statistical analysis packages
  • the cloud
  • “virtual data marts” allow data scientists to share existing data without replicating it
  • traditional role of IT— automating business processes — imposes precise requirements
  • Analytics has been more of an afterthought for monitoring processes
  • business and IT capabilities used to be stability and scale, the new advantages are based on discovery and agility
  • discovery and analysis as the first order of business
  • IT processes and systems need to be designed for insight, not just automation

• PayPal’s Mok Oh On What Is A Data Scientist? - Forbes

  The title is misleading: It’s not about what DS is. It’s rather a vision of the ideal solution.

  tags: datascience technology opinion

  • the old state and the ideal future state, which he calls “Analyst 1.0” and “Analyst 2.0,”
  • Analyst 1.0 as the state of maturity achieved by using the last generation of business intelligence tools
  • Analyst 1.0 has some coding skills, and perhaps writes an SQL query here and there
  • inflexibility of data warehouses and relational databases
  • Our current state of affairs, which we’ll call Analyst 1.5, finds us in limbo
  • two primary limitations: the immense size and variety of the data, and the complexity of the tools needed
  • Hadoop
  • to get value from big data, business analysts cannot simply be presented with a programming language
  • Analyst 1.5 is characterized by a disconnect between data scientists and the tools and systems in the more complex camp of programmers and computer scientists
  • caused data to be totally fragmented
  • Analyst 2.0 will have arrived when vendors and IT make analysis easy enough that a typical business user can conduct analysis entirely by themselves
  • Tools such as self-learning recommendations engines
  • demands new skills, such as a more precise focus on aberrant or statistically significant data in a stream, as well as better tools
  • somehow at some point you have to get your analytical inspection down to the equivalent of code level
  • what we’re trying to model is every person’s brain–at least the part of the brain that decides how to shop, when to shop, and what you want
  • we need to continue to mine for behavioral data, such as what people looked at before and after they made transactions
  • among the top pitfalls is the tendency to focus on a very small piece of data without occasionally stepping back
  • tendency to over-focus on technology
  • organizations are tempted to put the most technology-savvy person on the job, rather than the most business-savvy
  • computer scientists are not trained to ask the right business questions

• David MacKay: Information Theory, Pattern Recognition and Neural Networks: The Book

  tags: machinelearning books

  • Information Theory, Inference, and Learning Algorithms
  • 640 pages, Published September 2003
  • PDF (A4) pdf (9M) (fourth printing, March 2005)

• Cookbook for R » Cookbook for R

  HTML

  tags: r statistics programming

  • formerly named R Cookbook
  • It is not related to Paul Teetor’s excellent R Cookbook

• Big Data -- Why the 3Vs Just Don’t Make Sense -- TDWI -The Data Warehousing Institute

  When are we done defining big data? p.1

  tags: datascience bigdata opinion

  • settled on 3 Vs -- volume, variety, and velocity
  • another V: value
  • if big data is understood solely on the basis of these trends, it isn’t clear that it’s at all hype-worthy
  • if “big data” simply describes the volume, variety, and velocity of the information that constitutes it, our existing data management practices are still arguably up to the task
  • big data is hyped on the basis of its real or imagined outputs
  • a lot more interesting when you bring in ‘V’ for value

• No, Really, Some of My Best Friends Are Data Scientists

  When are we done defining data science?

  tags: datascience statistics opinion

  • the skills of a “data scientist” are those of a modern statistician
  • know how to move data around and manipulate data with some programming language
  • know how to draw informative pictures of data
  • Knowledge of stats, errorbars, confidence intervals
  • try to get people from different backgrounds
  • Great communication skills
  • a lot of what we teach The Kids now looks a lot more like machine learning than statistics as it was taught circa 1970, or even circa 1980
  • Everything I know about statistics I’ve learned without formal instruction
  • is not, in my experience, intrinsically hard for anyone who already has a decent grounding in some other mathematical science
  • mastering them really does mean trying to do things and failing
  • potentially hazardous. This is the idea that all that really matters is being “smart”
  • counter-productive for students to attribute their success or failure in learning about something to an innate talent

• Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics (Wiley & SAS Business): Bill Franks

  Bill Franks is Chief Analytics Officer at Teradata

  tags: datascience bigdata books

  • 27. April 2012
  • 336 Seiten

• The Big Question In Big Data Is...What’s The Question? | SmartData Collective

  tags: datascience opinion bigdata

  • Some folks like to confuse Hadoop with big data
  • Focus On the Questions To Ask, Not The Answers
  • The failure of data warehouses to provide real-time data led to the creation of data marts
  • Data marts failed to provide complete and updated and comprehensive views
  • existing solutions still don’t solve the problem. Why? The market and business environment have changed
  • Data moves from structured to unstructured. Sources exponentially proliferate. Data quality is paramount.
  • Real-time is irrelevant because speed does not trump fidelity. Quantity does not trump quality
  • Business questions remained unanswered despite the massive number of reports and views and charts
  • The big shift is about moving from data to decisions

• Information Theory, Pattern Recognition, and Neural Networks

  tags: machinelearning video lectures

  • Draft videos (editing incomplete)
  • Entropy and Data Compression
  • Shannon’s Source Coding Theorem
  • Inference and Information Measures for Noisy Channels
  • Introduction to Bayesian Inference
  • Approximating Probability Distributions
  • Clustering
  • Monte Carlo Methods
  • Gibbs sampling
  • Neural Networks
  • Other course materials - free online text book [Information Theory, Inference, and Learning Algorithms]

Posted from Diigo. The rest of my favorite links are here.